Data engineers out there, what does your day at job looks like, what tech stack and tools do you work on
Hey guys,
I'm a Java Backend SDE with 5 YOE and I want to switch to data engineering. I don't think software is for me, companies are expecting fullstack and I'm not good with frontend. Last 2 years, I had to switch twice and both times I got stuck in either legacy app which was complex and stressful to work on or a complex backend project where business logic was just out of this world. I think I got stuck in bad places since other people also experienced the same but this can happen anytime, so don't want this happening again in future.
I don't care about boring or repetitive work, I just want something that I can deliver, hence want to get into data engineering since there is big overlap, but I see that there are lots of tools for DE out there and no 2 data engineers do the same work, some are working on ETL, some in SQL.
So DE people in india, what tools and technologies do you guys mostly work on in your day to day tasks and did your work get easy and predictable with time.
Depends on the company, some uses azure stack warehouse, data factory etc . Some snowflake , data bricks . Majorly we code in python using spark , flink , airflow , camel , Kafka connecting all these tools
Thanks, and what about your work, did it get somewhat predictable and easy with time or at least bearable, in my case software is just getting too complex for me and I'm not good with frontend, everyone wants Fullstack now, so DE has a big overlap and looks like it can be easy compared to software. I don't care about boring or repetitive work, so let me know what your experience has been so far and your YOE.
4