Hello All. I am currently working for a retail analytics company as a Data Engineer + Data Scientist. As of now I have built a simple alerting system. It checks for alert subscriber and alert type during change calculation in ETL ( for example price change) and sends email if match is found. Its done via python scripts with somewhat reconfigurable alert sink ( SQS, email etc) and alert data ( count of product with change or all products list,). However its time to make it more scalable ( upto 1000 alerts per minute) and separate it out from main ETL pipeline. I always prefer simple solution. Any suggestion on any aspect of the process is welcomed. Thank you
Edit: There are updates below.
This is one of the most absurdly hilarious things I've seen at my current job.
A high-level marketing exec (very high in the company but not at the VP level, or we wouldn't be dealing with her direclty) is understandably very angry because she's getting calls at all hours on her mobile from our CritSit system that alerts on-call engineers of failures/emergencies in the hardware/software/infrastructure. There are a couple of hundred teams involved, and probably a thousand or two engineers in the on-call system. I'm in there, but in five years have only been called twice and once was an area of business I'm not part of. (Though I did respond, and collected some sweet sweet overtime.)
She is NOT one of the people who is on call. Hence, angry and verymuch WTF-fy. She has received a bunch of text-to-speech voicemails all sounding very urgent -- but they triggered her well-developed "this is not legit, I should notify security" detectors. (The company from time to time and without announcement sends phishing emails to different groups to find out who might fall for one. Awareness of the issue is "above average" here, I would say.
We went round and round over this. It looks an awful lot like a phishing attack. The grammar and spelling are characteristic of a well-known region of the world (not Nigeria) where a lot of suspect emails come from. It says it's reporting a problem with a particularly popular X.500 database -- also a product of the company -- but the team it references doesn't exist. "$X500 Prod" where $X500 is the name of the popular database system and "Production" is a common name for an instance of said database product. We just don't have an instance named that. It would be some kinda sophisticated attack that knows what critsit alerting system we use, and thinks it knows which database system we use internally -- but gets the instance name wrong and targets someone whose technical knowledge would probably fit on a single glossy brochure for whichever products she works with in marketing.
It took us days to find out who is responsible for the alerting system. It took a further two days to convince them to help us find the target's name somewhere in the on-call lists in the system. They kept insisting that it was some kind of phishing or social engineering attack and not coming from our system. Anyway, they relented and searched for her name and/or phone number.
... keep reading on reddit ➡
We’re using PRTG to monitor ~ 100 VMs and it works nicely, but our processes around the monitoring/alerting aren’t great.
Currently, we have a distribution list with 6 of our IT department who all get all the alert emails. The issue is that sometimes if there’s issues, we have everyone’s inboxes flooded with the same emails, and we don’t always know if an issue has been actively resolved or resolved by itself, i.e. if we get a disk space alert, has someone added disk space or has windows cleaned it up itself?
What do all you guys use, and what is your system around alerting and making sure that everything is dealt with correctly?
Made an AWS monitoring system with live flow monitoring, alerting, reporting, graphing etc...
The AWS monitoring product can be set up and integrated into your AWS network via the open source AWS lambda python function in about 20 minutes.
We are currently looking for early users that want to use it 3 months for free in exchange of providing us with your professional feedback as well as posting an unbiased review of our product on AWS Marketplace, G2 and Capterra.
Check it out on GitHub.
We are a the final stages of the development of our product, the AquaShield 2.0, that is a smart monitoring and controlling device for hydroponics and aquaponics. A year ago we meet with a Canadian cannabis community and they guided us what features do they need.
Sooo, we added some pretty cool new features and we made a video series about it. Check it out:
https://www.youtube.com/playlist?list=PL-W3TkTFLbcWkzkXtqF1vtNa9P0lgv7_L
First video is about all monitoring features and available sensors, 2nd is a short introduction to all controlling functions (simple relays and PWM outputs) and the last one is about creating automation (workflows for eg. time-based controlling you grow lights, ebb/flow cycle etc.) and alerting.
https://preview.redd.it/o1eykvtdgh171.jpg?width=1280&format=pjpg&auto=webp&s=aa8ae6d7772217a342ddcd12fc1bcd985b0bfa6d
I started this company and product to fit my needs, but it turned out that AquaShield is perfect for Cannabis cultivation! This device is built for this community, so let me know, if we hit the target :D
FYI: Soon we will launch on Indiegogo!
Hope you like it!
