Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pg_vector index support #7805

Open
34code opened this issue Dec 24, 2024 · 2 comments
Open

pg_vector index support #7805

34code opened this issue Dec 24, 2024 · 2 comments

Comments

@34code
Copy link

34code commented Dec 24, 2024

I'm unable to create an HNSW index using the following command:

CREATE INDEX ON com_ebay_items USING hnsw (title_vec vector_l2_ops);

I have citus installed alongside pg_vector (create extension if not exists vector)

Here is the error message I'm getting:

ERROR:  could not resize shared memory segment "/PostgreSQL.1153325950" to 1048576 bytes: No space left on device
CONTEXT:  while executing command on citus_worker.2.p2lw3ox7d6907doqtzz3crk12:5432
@onurctirtir
Copy link
Member

onurctirtir commented Dec 25, 2024

To speed-up the index build, hnsw tries to allocate maintenance_work_mem (seems set to 1G in your setup) amount of memory from shared memory if it decides that it can use multiple processes to build the index, so the error is actually caused by hnsw not Citus.

(Also note that for serial index builds, this is not the case because then hnsw won't try allocating memory from shared-memory but from the same backend's memory pool.)

This especially becomes a problem when your database is setup on a docker container.

https://www.timescale.com/learn/how-to-address-error-could-not-resize-shared-memory-segment
https://stackoverflow.com/a/56754077

If increasing shm-size doesn't help, please let us know again.

@34code
Copy link
Author

34code commented Dec 27, 2024

Hi I've successfully managed to increase shm_size to 8g using docker's daemon.json but this error comes back with a larger number instead of 1048576 bytes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants