| 9 Jan 2021 |
 Paul Masurel (Gitter) | if you tend to commit independently from your change (ie: on a different thread, taking a lock on the indexwriter) | 00:22:40 |
| Paul Masurel (Gitter) | you can send both operation at the same time to ensure that your upsert looks like a transcation | 00:23:28 |
| Paul Masurel (Gitter) | and at no time someone sees your index in a state where you have two docs with the same id, or 0 doc with the same id | 00:23:52 |
| Paul Masurel (Gitter) | if you do delete , add , commit_ in the same thread, you will get that property too of course. | 00:24:38 |
 Follpvosten | So my only operation is delete, then re-add, and currently I'm acquiring a writer for each of these | 00:26:44 |
| Follpvosten | Ah, and so using IndexWriter::run does have a benefit, interesting | 00:27:06 |
| Follpvosten | I kinda assumed it didn't really matter since it's all sent on commit() | 00:27:36 |
| Follpvosten | When it comes to threads, I think there's an await-point between the change being saved in the database and updating the search index, so it might thread-hop there in theory | 00:29:22 |
| 11 Jan 2021 |
| Paul Masurel (Gitter) |
So my only operation is delete, then re-add, and currently I'm acquiring a writer for each of these
If it is a service, please make a writer once and for all and reuse it. | 10:24:36 |
| Paul Masurel (Gitter) | it will require locking | 10:24:39 |
| Follpvosten | What defines a "service"? My wrapper around the index, reader etc. will regularily jump between threads, so it needs to be Send + Sync | 15:04:35 |
| Paul Masurel (Gitter) | I mean an ever running program as opposed as a CLI | 15:30:22 |
| Paul Masurel (Gitter) | Your use an arc< rwlock> | 15:31:02 |
| Paul Masurel (Gitter) | You could use* | 15:31:15 |
| Paul Masurel (Gitter) | The best option requires a lot more plumbing. | 15:31:47 |
| Follpvosten | Yeah then it is a service (which is why an in-memory index works well for me) | 17:33:30 |
| Follpvosten | I think I'll want to benchmark how much I'd gain from an Arc<Mutex<>> there | 17:34:11 |
| Follpvosten | (RwLock doesn't really make sense as it's read optimized and the writer will only ever be used as mut) | 17:34:45 |
| Follpvosten | Fact is, I don't expect many writes at all, at least at first | 17:35:15 |
| Follpvosten | So keeping the writer around might be an optimization for later | 17:35:39 |
| 12 Jan 2021 |
| Paul Masurel (Gitter) | IndexWriter::add_document is not &mut | 02:06:04 |
| Paul Masurel (Gitter) | I strongly recommend using an Arc<Mutex<...>> and keeping the same IndexWriter | 02:08:04 |
| Paul Masurel (Gitter) | this is not just about performance | 02:08:13 |
| Follpvosten | In reply to @gitter_fulmicoton:matrix.org
IndexWriter::add_document is not &mut Well I need to commit immediately, always | 03:18:53 |
| Follpvosten | In reply to @gitter_fulmicoton:matrix.org
I strongly recommend using an Arc<Mutex<...>> and keeping the same IndexWriter But alright, I'm not against it | 03:19:09 |
| 16 Jan 2021 |
|  rajyogi joined the room. | 02:26:24 |
| 17 Jan 2021 |
|  Matthew (Gitter) joined the room. | 12:17:07 |
| * Matthew (Gitter) waves | 12:17:08 |
| Matthew (Gitter) | fulmicoton: i just saw https://twitter.com/fulmicoton/status/1350776034094813184?s=20 | 12:17:22 |
| Matthew (Gitter) | i think the problem running tantivy in a browser via wasm is the threading; do you think that is a tractable problem? | 12:18:17 |
