1) develop.sh provides a direct interface to invoke manage.py. To invoke manage.py, run:
2) To get a list of manage.py commands, run:
Those are the two ^
prabal joined the channel
sumedh has quit
sumedh joined the channel
sumedh has quit
prabal
https://tickets.metabrainz.org/browse/BB-50 I am looking at this. A graph is to be plotted. A simple google search showed me a number of different javascript libraries that would get the work done. Just wanted to ask if anyone has done something similar to this before.. ?
BrainzBot
BB-50: Add editor activity graphs
sumedh joined the channel
sumedh has quit
sumedh joined the channel
sumedh
prabal: Hi! I have used plotly.js for generating graphs in the project I did during my internship
Please let me know if I can help in any way
sumedh has quit
Nyanko-sensei joined the channel
D4RK-PH0_ joined the channel
Nyanko-sensei has quit
sumedh joined the channel
D4RK-PH0ENiX has quit
prabal
Umm the code is in nodeJs/reactJs I don't think I can use python libraries for that
sumedh
prabal: Plotly.js is a javascript library
prabal
oh sorry
I read it as py
my bad
@sumedh :P
c1e0 has quit
sumedh
No problem. Please let me know if I can help in any way
sumedh has quit
prabal
Yeah sure. I'll have a look at it. Thanks :D
shivam-kapila joined the channel
shivam-kapila
sumedh: I couldn't get what exactly is missing in documenation. If you could explain I could possibly update it alongwith other additions I am making
I'll do soon, I'm shooting a bunch of abusers first
Matthew_ joined the channel
Matthew_ is now known as Guest19787
MajorLurker has quit
MajorLurker joined the channel
ruaok
iliekcomputers: any idea what we should make of these LB inbound queue spikes?
iliekcomputers
ruaok: which queue?
ruaok
see MB alerts on telegram.
it is short lived spike that goes away quickly. but we've been getting more of them.
iliekcomputers
Interesting.
Could it be because of increased usage? :D
ruaok
that would be nice, but I'd be concerned about the pipeline filling up so fast.
it might just be triggered by someone doing an import...
iliekcomputers
Mhmm
It triggers at 1000 messages
We haven't deployed anything LB related in prod recently.
Have we done any rabbitmq upgrades recently.
?
ruaok
not that I know of. do you zas?
zas
I'm not aware of any recent upgrade
Darkloke has quit
alastairp
zas: when you have time, I would like to ask you some questions about your abuse identification/blocking pipeline. we're seeing more people trying to attack freesound with SQL injection attacks. we need to work out a way to identify and block these users too
iliekcomputers
No new influx writer errors in sentry that I see
Sidenote: there's a few errors in sentry that happen so often, we really should fix them
yup, perhaps some scripts submitting stuff in batches
alastairp: sure, what do you want to know?
alastairp
zas: thanks. a few questions: for MB, are abusers just IP addresses making more than x requests in time y? Do you have any more detailed tracking (e.g. many POST requests in a short time)? Do you have any tools to try and identify these patterns, or do you just look at logs from time to time?
do you just have a manual block-list?
zas
abusers are not just about x requests vs time (because we rate limit most of the stuff anyway), when I'm suspecting excessive traffic (based on grafana graphs+mb top stuff), I audit logs looking for patterns
based on UA/IP (lookup in various blacklist services) plus nature of the requests (like same request over and over, and/or obvious ignorance of response codes), I decide what is the best action
alastairp
right. thanks
good point about looking up in blacklist. I should do that for this malicious IP too
zas
then I manually block offending IPs or add the UA to black lists we have at nginx level
alastairp
it looks like I need to improve our log processing as well
for now it seems like it was only one IP address. I've stopped seeing error requests come in with sql injection parameters
Mr_Monkey
prabal: I had a look previously at different charting libraries examples, and I also liked amcharts. On top of solid standard graphs, I can think of some cool uses of chord diagrams for displaying relationships between entities, and displaying an Author's timeline (https://www.amcharts.com/demos/timeline/). There's good maps as well which could be nice to display all authors' country of origin as a heatmap.
zas
if your traffic isn't too heavy, you can easily write a log parser that check patterns for you, but at the end, you have to take decisions... I guess we could use some AI-fueled software to help, but blocking blindly can lead to other problems. Also you need to differentiate legit bots (ie. google crawlers) from dumb spiders (we have bots crawling website from data available through web service...)
c1e0 has quit
c1e0 joined the channel
alastairp
yeah, right. we were just talking with some others about that
for now, we can probably check for keywords that look like sql injection attacks and report them
so that we can tweak some rules
ruaok
invoices paid. nice and timely, thanks everyone!
c1e0 has quit
prabal
@Mr_Monkey Thanks I will have a look at amcharts :)
ruaok
Mr_Monkey: thank you for editing the BB GSoC ideas page.
can you please take a look at what remains to be done and see if this is something we can package into another GSoC project to get the project finished and depoloyed?
yvanzo
Yes it is, I will add it to MB ideas.
ruaok
thank you.
can you please do that today or no later than tomorrow?
ruaok assigns OTHER-348
shivam-kapila
ruaok: Hi. I am GSoC aspirant and would like to contribute to ListenBrainz. I would like to know the possible roadmap of the goals that are under current focus. Kindly let me know when you would be free for a discussion. :) Thanks