DancingNugget
DancingNugget

Is Data Engineering less stressful compared to software engineering

Hi Guys,

I'm an Java SDE and I have had bad luck twice now, I got stuck in a company with a project which was super complicated and many people left from there, another company I joined was a service based company where I got stuck in a legacy java project and management having no experience in tech and creates pressure to deliver quickly.

I am sick of this for the last 2 years now, I want to change my field, is data engineering less stressful compared to software in terms of work complexity.

And what are the most used technologies that companies use for data engineering, like for backend development, spring boot java and node.js are mostly seen. For data engineering what are those technologies.

4mo ago
4.2Kviews
Find out if you are being paid fairly.Download Grapevine
JazzyWalrus
JazzyWalrus
TCS4mo

In DE, some projects would require you to have lot of domain knowledge(specific to the client's operations). Until you acquire that. Those kind of projects will be stressful.

Other than that most of the times it will be repeative work. If not it will be kind of medium level leet code problems, like if n inputs from m no:of tables needs to be transformed into k outputs of L no:of tables. Somethings the i/p to o/p transformation can get really complex.

If a project have bad architecture i.e, the data fragmentation is too much. Instead of 20 to 30 jobs across 2 or 3 layers/stages. It is split across 50 to 100+ jobs across 4 to 6 stages.

Those kinds of projects will be really stressful.

DancingNugget
DancingNugget

So in other words there's no guarantee that the project will be less stressful, I'll try to avoid any small company in that case and especially an Indian small service based company, licking clients a** and putting all pressure on developers are their favourite hobbies, it's a meat grinder that's for sure.

SnoozyCoconut
SnoozyCoconut

De is boring work for setting up the same pipelines,no innovation nothing

DancingNugget
DancingNugget

Don't really care, I just want less stressful work, don't care if it is repetitive, as long as it's easy compared to software. Do you know what tech stack should I focus on for data engineering if you are aware

SleepyPickle
SleepyPickle

It's mostly like a sanitation worker. You won't know what shit you have to clean up on a given day. All fancy tools and frameworks but imo, it's like the administration department playing around with their excel sheets.

DancingNugget
DancingNugget

Okay, do we have same agile methodology in Data engineering as well, like in software we have to give estimates and then deliver the work under those, if we can't then manager eats out brain. Do we have similar situation in data engineering also, I just wanted to understand if it is easy compared to software engineering

PeppyNarwhal
PeppyNarwhal

Not sure whether DE is less stressful or more but Your professional stress mostly depends on the project and team you are assigned to.

Discover more
Curated from across
Software Engineers

Data Engineering to SWE

Hi All,

I have 6 YOE. Initial 2.5 years I worked as Backend developer in Python, Django, AWS and Little bit JS. After that all of my experience is in Data Engineering mostly in GCP (have few certifications as well.) Currently I am worki...