Skip to content

lookups should throw an error if the number of input keys is too large #842

Description

@jorisdral

When calling Database.LSMTree.lookups with large-sized batches of keys, @jasagredo was seeing unexpected NotFound lookup results. The offending code is a typo in a bit-mask that we use while querying bloom filters. This typo effectively limited the number of bloom queries we could perform on a single run to 4096 (hexadecimal: 0xfff), while the limit should have been 65536 (hexadecimal: 0xffff). So under high lookups workloads, this limit would be hit and hence the erroneous lookup results.

Also: this limit should probably not be implicit. Instead, lookups should return an error if the number of input keys ever exceeds the limit of what we can handle in lsm-tree.

Originally posted by @jorisdral in #841 (comment)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinglsm-treeRelated to the lsm-tree librarypriority: mediumMedium priority

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions