DBTNG provides transaction support. We'll see a big improvement on insert performance if we wrap our inserts into a single transaction, as much as possible. Particularly important for d.o, since we're going to be doing probably in the tens of millions of inserts when we build the data.

Comments

sdboyer’s picture

sdboyer commented
Project: Version Control API -- Git backend » Version Control API
Component: Code » API module
Priority: Critical » Major
Issue tags: -git sprint 8 +git sprint 9

Whoops, wrong queue. We probably could earn a fair bit from this, but hopefully we won't NEED to experiment with it until sprint 9.

sdboyer’s picture

sdboyer commented
Issue tags: -git sprint 9 +git sprint 10

Potential performance issue, retagging to sprint 10. Otherwise will leave it for phase 3.

sdboyer’s picture

sdboyer commented

Gonna be revisiting this next week when we finally start calculating just how much volume util can handle.

eliza411’s picture

eliza411 commented
Status: Active » Postponed
Issue tags: -git phase 2 +git phase 2 leftovers
marvil07’s picture

marvil07 commented

Seems like we have enough hardware for now :-p, but yes we need to do this, but probably after generic-parser-framework branch merge back to mainline.

marvil07’s picture

marvil07 commented
Version: 6.x-2.x-dev » 7.x-1.x-dev
Issue tags: +vc-next

During triage.