LanceDB table existence check fails when more than 10 memory tables exist #85

Bug Description

When using SimpleMem with more than 10 LanceDB tables, reopening an existing memory table can fail with:

Table '<table_name>' already exists

The issue appears to come from the table existence check in LanceDBVectorStoreBackend._init_table():

if self.table_name not in self.db.table_names():
    self.table = self.db.create_table(self.table_name, schema=schema)
else:
    self.table = self.db.open_table(self.table_name)

In recent LanceDB versions, table_names() is deprecated and defaults to returning only 10 tables.

As a result, once the database contains more than 10 tables, an existing table that is not included in the first page may be incorrectly treated as nonexistent. SimpleMem then calls create_table(), which fails because the table actually already exists.

Reproduction

1. Create more than 10 SimpleMem memories with different table_name values.
2. Restart the process.
3. Reopen one of the tables that is not returned in the first 10 results of db.table_names().
4. SimpleMem attempts to create the table again.

Observed error:

Table 'todo5-test-memory' already exists

Removing the other memory tables so that fewer than 10 tables remain makes the same reopen operation work correctly.

Environment

• SimpleMem: 0.3.0
• LanceDB: 0.37.1
• Python: 3.13

Root Cause

LanceDB's current Python API defines table_names() with a default limit of 10.

LanceDB already implements full pagination for membership checks through Connection.__contains__() / _all_table_names().

This is also related to the table listing/pagination behavior discussed in lancedb/lancedb#2727.

Suggested Fix

Replace:

if self.table_name not in self.db.table_names():

with:

if self.table_name not in self.db:

The latter iterates through all table-list pages instead of checking only the first 10 tables.

After applying this change locally, an existing table can be reopened correctly even when the database contains more than 10 tables.

I would be happy to submit a small PR for this fix if desired.