How do you manage event stream, external product/in house?
Same for Data pipelines
Same for reporting
In my previous org the events were sent from the backend and the app fired them to a firebase server for the event stream. I am not aware of data pipelines and reporting method
Got it, so mostly reliant on the developer to put the events?
Yep
In our company, we send events via Segment from all clients (web, app, backend).
From Segment, we send to multiple places including S3 and Mixpanel.
From S3, we have pipeline built to ingest to Snowflake on regular intervals. From Snowflake, we do business reporting via Tableau.
We also create monitoring dashboard on Mixpanel.
I would assume this is standard setup at most mid to large startups.
You can use Jitsu to route events from frontend and send it to snowflake and all other tools such as amplitude. Use snowflake data to build visualisations or metrics.
I work for a growing startup in Bangalore. The data team size is very small with no data lead. Not a single query we hit the DB gives any result. Owing to cost cutting my firm has cut down the instance size and they're planning to downgr...
Hi community,
Context: In my current company, we have a data-pipeline, which in short works like this: we get raw events from Kafka dumped in S3. We run a batch job (Airflow), this job essentially picks up the raw jsons in s3, enforces...
AMA. How does a day to day pan out for a data scientist, Who is data science not really for, Does FOMO really affect my career in DS
