[Help] Trying to upgrade to recent RocksDB

I’m trying to upgrade RocksDB to a more recent version and I have the following error when compiling the code I have at the moment https://pastebin.com/raw/W0KaSCjJ. What is weird is that the part that doesn’t compile is exactly the same as in the current version of tikv/rocksdb and it fails on rocksdb/db.h at 6.29.tikv · tikv/rocksdb · GitHub.

What is annoying is that it complains about 2 things as you can see, a conversion issue and the fact there is some deprecated code. But the same code in your repo with the exact same compiler doesn’t fail. So any idea what is going on ? What is also surprising is that vscode sees the same conversion error in my repo but not in yours when both files are similar there. It happens after it parses some Makefile or CMakeFile but I can’t see where it comes atm. I also try to run the exact same cc1plus command and it fails the same. Any guidance would be much appreciated, @tabokie can you help on this ?

Hi, I am on leave this couple of weeks. But I take a look at your errors, it should be related to these PRs:

Hey @tabokie sorry I just realized I didn’t respond but yeah it helped. I’m slowly making some progress and will keep asking you on this thread if I’m stuck again. Thanks

Hey @tabokie, I’m still trying to make it work. Can you tell me what am I supposed to do to confirm it’s working as expected before I create a PR ? I’m running make check in a docker container, which is taking years tbh. Is there anything else I can do ? Would it be better to create a PR and have you allow it to run on your CircleCI somehow ?

Yeah, you can do that. Normally we just make sure the code can compile and is formatted before filing a PR.

Can you create a new branch or tell me which branch I should create the PR on ? I suppose it doesn’t make sense to create it on 6.29.tikv and you have a dev branch somewhere.
I only have a few tests that fail now. It will be easier to talk about them when the PR is created

t/run-compact_files_test-CompactFilesTest.FilterContext
ALL db_kv_checksum_test . DbKvChecksumTestMergedBatch

3.151 FAIL t/run-db_write_test-DBWriteTestInstance-DBWriteTest.PostWriteCallback-0
3.141 FAIL t/run-db_write_test-DBWriteTestInstance-DBWriteTest.PostWriteCallback-2

4.371 FAIL t/run-listener_test-EventListenerTest.OnSingleDBSubcompactionTest

If I execute one of DBWriteTest.PostWriteCallback-? it succeeds one or multiple times then crashes

@tabokie ^

@harpocrates That’s really impressive work! Thanks for putting in the time.

For the PR, you can create one against 6.29.tikv. If needed I will make some adjustments to our internal CI so that your changes can compile there. Once the changes are approved, I will manually create a branch with your changes in it as the new default branch.

And BTW sorry for the delay, I am not paying much attention to the forum. If there’s any problem you can ping me on GitHub more easily.

1 Like