iliekcomputers: So, inside db folder, we've all the files (functions) that perform some operation on the database. Right?
2018-10-18 29118, 2018
iliekcomputers
Yes
2018-10-18 29112, 2018
zas
ruaok: we'll be discussing about ways to upgrade solr tonight with samj1912 (related to security warnings reported by github)
2018-10-18 29100, 2018
ruaok
> Sorry, I've been sleeping for the past two days
2018-10-18 29108, 2018
ruaok
LOL, yes. I got 2 nights of 12 hours in myself.
2018-10-18 29114, 2018
code_master5
iliekcomputers: Whenever you begin some transaction using the connection object, you need to use the same connection object to execute a query (which is part of that transaction).
2018-10-18 29117, 2018
ruaok
zas: great.
2018-10-18 29101, 2018
code_master5
iliekcomputers: currently, all the function create their own connection object and close it before returning the result.
2018-10-18 29107, 2018
code_master5
*function
2018-10-18 29111, 2018
code_master5
*functions
2018-10-18 29102, 2018
iliekcomputers
Ok
2018-10-18 29102, 2018
iliekcomputers
What's the problem?
2018-10-18 29119, 2018
code_master5
iliekcomputers: the problem is that, in order to make the codebase support transactions, we need to rewrite the functions (all of them, which execute some query on db).
2018-10-18 29130, 2018
code_master5
iliekcomputers: The idea is to pass the same connection object to each function.
2018-10-18 29114, 2018
iliekcomputers
code_master5: the ticket in question only wants to do the dumps inside a transaction
2018-10-18 29102, 2018
iliekcomputers
If you begin a transaction before a dump, create the dump, then rollback, the dump is guaranteed to be consistent.
2018-10-18 29149, 2018
code_master5
iliekcomputers: The function for json dumps requires to extract licenses, distinct entities, reviews etc to be extracted from the database. What if, while the data for dumping is being extracted and some user adds or deletes the reviews?
2018-10-18 29110, 2018
CatQuest
make readonly at the time?
2018-10-18 29123, 2018
CatQuest
should only be a short time
2018-10-18 29124, 2018
alastairp
code_master5: for the most part you can assume that that won't happen
alastairp: Then why do we need to work on this issue in the first place? 🤔
2018-10-18 29136, 2018
alastairp
because best practices dictate that doing this kind of operation in a transaction is a good idea
2018-10-18 29147, 2018
ruaok nods
2018-10-18 29152, 2018
ruaok
to make the database dump consistent.
2018-10-18 29109, 2018
ruaok
so that the timestamp in the dump EXACTLY represents what is in the dump.
2018-10-18 29114, 2018
iliekcomputers
Oh and code_master5, you shouldn't wait for me or anyone to come online before you ask your questions, just ask, people will help if they're around and can.
2018-10-18 29119, 2018
code_master5
iliekcomputers: Well sorry I'm new here. I didn't know that. I asked, but someone pointed to you. 😅
2018-10-18 29151, 2018
ruaok
no need to be sorry. we're just showing you round our community. :)
2018-10-18 29123, 2018
code_master5
ruaok: Ok. 😅
2018-10-18 29117, 2018
code_master5
iliekcomputers: I can't understand how the dump will be consistent if we don't perform the underlying operations in a transaction.
2018-10-18 29148, 2018
ruaok
iliekcomputers, alastairp: seemingly unrelated, but not really:
bitmap: how's the relationships stuff coming along? :)
2018-10-18 29122, 2018
ruaok
some background reading on my current music industry noodling. start with beer. :)
2018-10-18 29128, 2018
reosarevok
ruaok: wait, but isn't everyone already a snobbish hipster about music?
2018-10-18 29145, 2018
ruaok
apparently not quite enough yet.
2018-10-18 29141, 2018
iliekcomputers
code_master5: I have a feeling that new changes won't be seen in the transaction once you've begun it
2018-10-18 29157, 2018
iliekcomputers
But I'll have to look at what sqlalchemy does exacrly
2018-10-18 29100, 2018
ruaok
iliekcomputers: correct.
2018-10-18 29116, 2018
code_master5
iliekcomputers: Yeah. Me too.
2018-10-18 29128, 2018
ruaok
that is a function of the DB, not of sqlalchemy. it is called transaction isolation.
2018-10-18 29129, 2018
code_master5
iliekcomputers: Well, although you're starting a transaction, but you're also starting new connections. And the connections are the ones which are extracting data. A transaction object can be associated with a single connection only. So!.. https://docs.sqlalchemy.org/en/latest/core/connec…
2018-10-18 29138, 2018
code_master5
other connection objects are independent of the transaction.
2018-10-18 29155, 2018
iliekcomputers
code_master5: what i was trying to say was the things that the new connections change won't be reflected in the transaction you've opened
2018-10-18 29153, 2018
code_master5
iliekcomputers: Yes! you're right. But that only applies if the connection object related to that transaction performs the reading. The point is that the dump is being created from the data read by independent connections (and they've nothing to do with the transaction).
2018-10-18 29138, 2018
iliekcomputers
code_master5: oh
2018-10-18 29143, 2018
iliekcomputers
Then that should be fixed
2018-10-18 29117, 2018
iliekcomputers
Once a transaction is created, you should only read data from that transaction's object
2018-10-18 29138, 2018
code_master5
iliekcomputers: Exactly!
2018-10-18 29157, 2018
iliekcomputers
Great :)
2018-10-18 29141, 2018
code_master5
otherwise, the independent connections are just equivalent to some other connection that might be updating database at the same time. 🤔
2018-10-18 29143, 2018
iliekcomputers
Yeah
2018-10-18 29107, 2018
code_master5
iliekcomputers: So now... It is like we've to pass the same connection object to every function which accesses the database (atleast those which're being invoked by dump function). But then, we cannot use context managers to perform the operations.
2018-10-18 29129, 2018
iliekcomputers
Not every function
2018-10-18 29138, 2018
iliekcomputers
Just the ones that the dumps use
2018-10-18 29149, 2018
code_master5
iliekcomputers: Yeah.
2018-10-18 29158, 2018
iliekcomputers
Context managers?
2018-10-18 29113, 2018
code_master5
iliekcomputers: Python context managers..
2018-10-18 29135, 2018
code_master5
with db.engine.connect() as connection: ....
2018-10-18 29150, 2018
iliekcomputers
Just pass the connection to the other functions
2018-10-18 29116, 2018
code_master5
iliekcomputers: That will not work.
2018-10-18 29131, 2018
iliekcomputers
why not?
2018-10-18 29143, 2018
code_master5
iliekcomputers: Because context managers automatically close connection before returning the results.
2018-10-18 29101, 2018
code_master5
iliekcomputers: We need to close connection only after transaction is over.
2018-10-18 29138, 2018
code_master5
So, that means, falling back to try... except.. finally..
the with clause won't close the connection just because you pass to a different function.
2018-10-18 29110, 2018
iliekcomputers
ruaok: those articles are interesting.
2018-10-18 29117, 2018
iliekcomputers
>Sold on the basis of its quality and non-corporate pedigree, the once-niche market now turns over around $20bn each year – more than the entire global recorded music business.
2018-10-18 29123, 2018
D4RK-PH0ENiX has quit
2018-10-18 29146, 2018
iliekcomputers
Whaaaat?
2018-10-18 29129, 2018
discopatrick joined the channel
2018-10-18 29123, 2018
ruaok
iliekcomputers: the music industry is tiny. a wag of google's tail and it could be all destroyed.
2018-10-18 29143, 2018
ruaok
but, the lessons are key. community, collaboration and pride in your product.
