"Given key was not present in the dictionary" after migration to 2.0

[Szmiglo]

Hello,
I have a problem with (pretty basic) selecting data with FirstOrDefaultAsync. The code was of course working on 1.1.0 of EFCore. I'm using PostgreSQL db provider. After migrating to 2.0.0-preview1-final I'm getting exception:

fail: Microsoft.EntityFrameworkCore.Query[1]
      An exception occurred in the database while iterating the results of a query.
      System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
         at System.ThrowHelper.ThrowKeyNotFoundException()
         at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
         at Microsoft.EntityFrameworkCore.Query.Internal.QueryBuffer.System.IDisposable.Dispose()
         at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<ExecuteSingletonAsyncQuery>d__23`1.MoveNext()
System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
   at System.ThrowHelper.ThrowKeyNotFoundException()
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryBuffer.System.IDisposable.Dispose()
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<ExecuteSingletonAsyncQuery>d__23`1.MoveNext()

Code that selects data:

client = await _db.Set<Client>()
  .Include(x => x.Contacts).ThenInclude(x => x.ApplicationUser)
  .Include(x => x.Contacts).ThenInclude(x => x.Interests)
  .Include(x => x.Offers).ThenInclude(x => x.OfferProducts).ThenInclude(x => x.Product)
  .FirstOrDefaultAsync(x => x.SubscriberNr == 13);

My model:

    public class Client : EntityBase
    {
        public string FirstName { get; set; }
        public string LastName { get; set; }
        // ...
        public int SubscriberNr { get; set; }
        public virtual ICollection<Offer> Offers { get; set; }
        public virtual ApplicationUser EmployeeAdded { get; set; }
        public virtual ICollection<Contact> Contacts { get; set; }
        public virtual ICollection<Payment> Payments {get; set;}
   }
    public class Offer : EntityBase
    {
        public string CodeName { get; set; }
        // ...
        public OfferStatus Status { get; set; }
        public virtual Client Client { get; set; }
        public virtual ICollection<OfferProduct> OfferProducts { get; set; }
    }

    public class OfferProduct : EntityBase
    {
        public virtual Offer Offer { get; set; }
        public int OfferId { get; set; }
        public virtual Product Product { get; set; }
        public int ProductId { get; set; }
        public DateTime? DueDate { get; set; }
        [DataType(DataType.Currency)]
        public decimal Price { get; set; }
    }

    public class EntityBase
    {
        [DatabaseGenerated(DatabaseGeneratedOption.Identity)]
        [Key]
        public int Id { get; set; }
        public DateTime DateAdded { get; set; }
        public DateTime? DateUpdated { get; set; }
        public bool IsDeleted { get; set; }
    }
public class Contact : EntityBase
    {
        public string Description { get; set; }
        public int? ClientId { get; set; }
        public virtual Client Client { get; set; }
        public virtual ApplicationUser ApplicationUser { get; set; }
        public virtual ICollection<Interest> Interests { get; set; }
        public ContactStatus ContactStatus { get; set; }
    }

Please let me know if you want addidional info.

Interesting thing: when I'm using code below instead of previous one I'm getting Client object from db, but with null in Client.Offers.OfferProducts field (included by ThenInclude)

 client = _db.Set<Client>()
                    .Include(x => x.Contacts).ThenInclude(x => x.ApplicationUser)
                    .Include(x => x.Contacts).ThenInclude(x => x.Interests)
                    .Include(x => x.Offers).ThenInclude(x => x.OfferProducts).ThenInclude(x => x.Product)
                    .Where(x => x.SubscriberNr == 13)
                    .Select(x => new Client()
                    {
                        Id = x.Id,
                        // all the fields of Client entitty
                        Offers = x.Offers,
                        EmployeeAdded = x.EmployeeAdded,
                        Contacts = x.Contacts,
                        Payments = x.Payments
                        
                    })
                    .ToList()
                    .FirstOrDefault();

Further technical details

EF Core version: 2.0.0-preview1-final
Database Provider: Npgsql.EntityFrameworkCore.PostgreSQL (2.0.0-preview1)
Operating system: Windows 10 (1607)
IDE: Visual Studio 2017.3

[anpete]

@Szmiglo I tried to repro this but have been unsuccessful so far. Would you be able to put together a small repro project for this?

[anpete]

We made a mitigating fix that should prevent this issue.

[colltoaction]

Can confirm this is affecting to me too with SqlServer, .NET Core, Windows 10. I haven't been able to test the fix in preview2 because of #9129

[smitpatel]

@tinchou - Does this work for you now? Given that #9129 is just new pattern.

[colltoaction]

Will let you know as soon as I can migrate the code

…

On Mon, Jul 10, 2017 at 10:27 AM Smit Patel ***@***.***> wrote: @tinchou <https://github.com/tinchou> - Does this work for you now? Given that #9129 <#9129> is just new pattern. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#8817 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABeg9IvOxZJ33FDuLTf3e8c3VeTpkvXxks5sMl8dgaJpZM4N2hSU> .

[colltoaction]

Still seeing this error. The important part of the stack trace seems to be:

System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
   at System.ThrowHelper.ThrowKeyNotFoundException()
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryBuffer.System.IDisposable.Dispose()
   at Microsoft.EntityFrameworkCore.Query.QueryContext.Dispose()
   at Microsoft.EntityFrameworkCore.Query.Internal.AsyncLinqOperatorProvider.ExceptionInterceptor`1.EnumeratorExceptionInterceptor.Dispose()
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<ExecuteSingletonAsyncQuery>d__23`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at XXXXX.Controllers.YYYYYYController.<Index>d__5.MoveNext() in C:\Users\martin\XXXXX\Controllers\YYYYYYController.cs:line 41

Similar to @Szmiglo, I have a complex query with multiple Includes and ThenIncludes.

My csproj's EF dependencies are:

<PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="2.0.0-preview2-final" />
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="2.0.0-preview2-final" />

[smitpatel]

@tinchou - This was fixed in 2.0 release and you would still hit exception in 2.0.0-preview2-final packages.

[colltoaction]

But those aren't published on Nuget yet, right?

…

On Fri, Jul 14, 2017, 10:20 AM Smit Patel ***@***.***> wrote: @tinchou <https://github.com/tinchou> - This was fixed in 2.0 release and you would still hit exception in 2.0.0-preview2-final packages. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#8817 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABeg9HvTrhs_jTkBnXi8VdariI6cYnpKks5sN6NHgaJpZM4N2hSU> .

[smitpatel]

That is correct. they are not in nuget yet but you could try nightly builds

[mrlund]

@anpete in 43e2ae1 you indicate that the conditions are unsure, and if you're interested I may have found one thing that causes it; if the loaded entities are missing the related entities loaded with .ThenInclude() then this happens. If related data is available for all the entities then it's good.

@tinchou could you try making sure that all loaded entities have the specified related entities?

[colltoaction]

@mrlund do you mean this?

If I had only

.Include(u => u.XXX).ThenInclude(i => i.YYY)

Now I have

.Include(u => u.XXX)
.Include(u => u.XXX).ThenInclude(i => i.YYY)

If that's your suggestion, then it doesn't fix it.

---

@smitpatel is there a nightly feed? are there instructions to set it up?

[emanuelmarques]

im getting this error while installing a nuget package (autofixture). Any solutions on sight?

[ajcvickers]

@emanuelmarques This issue was fixed and released in 2.0.0. If you are still seeing a problem using the 2.0.0 release, then please open an new issue including complete details, versions used, and a code listing or project that reproduces what you are seeing.

[emanuelmarques]

@ajcvickers the problem was with autofixture itself. it didnt support .netcore yet. The new RC does though
Thanks anyway :)