SquishyQuokka
SquishyQuokka

Is Data Science a bad bet for your career longevity?

Talking to my nephew nowadays seems like the AI bug has gotten to him. Even going back during the early parts of my own career and even undergrad, ML and stats was still a fledgling field. Most of the smart folks went into SDE roles at top tech.

I still remember that even back then, hiring for roles like these were extremely theoretical. You would test even experienced candidates with undergrad topics which was slightly outdated tbh because most of the functions were abstracted away and it is obvious that you would not remember the matrix approach to finding weights for a simple linear regression model.

Has this changed now? Because I don't think interviews like that actually evaluate the accumulated wisdom that experienced professionals bring to the table.

I mean it is generally a waste of time for anyone to brush up manual derivations or remember SQL niche intricacies. Usually it is better to test for the 80% knowledge that is actually useful on a day to day basis. I mean if the main things that companies test in experienced candidates is something that can be rote-memorized from a book then they are not testing the right things no?

What do you all think? Just wondering about this for some time now especially considering the hype around AI?

11mo ago
App Promo
ZoomyMuffin
ZoomyMuffin

Agree 100%.

That Matrix weight linear algebra for vectors dot product is so cringe. I was interviewing for WorldQuant and he made me look very stupid. My confidence got shattered and there was no comeback possible.

10 Saal se Excel pe Sumproduct kiye Hai, why do you want me to know past things? It is a stupid rulebook to interview senior folks who have years of experience in the own field.

ZoomyMuffin
ZoomyMuffin

I have my own interview playbook now.

If I am asked something that I can easily get from my old notebooks, I just say

"I have not worked on this specific concept since <2012>, so I won't attempt to answer confidently. But I have pivoted many times in my role from x-y-z, and been a very quick learner as you can see from my CV.

Only if you feel from your past experience that concept is very tough for most of your other joiners and you would reject someone because of that concept,

I could brush up and come back for next round and you could test me if that is so important to my role "

SwirlyPretzel
SwirlyPretzel
Google11mo

This is very similar to my answer and it usually works. Recruiters and Hiring Managers are aware that knowing every archaic concept is an unfounded expectation.

QuirkyMarshmallow
QuirkyMarshmallow

Everything changes along time t. Do not compare a situation at time T with T+T1

SwirlyPretzel
SwirlyPretzel
Google11mo

Hmm, what do you mean exactly? I agree with this mostly tbh.

QuirkyMarshmallow
QuirkyMarshmallow

During his graduation of days of OP SWE was booming and Data science was nascent does not mean both situations cannot revert due to external conditions.

QuirkyMarshmallow
QuirkyMarshmallow

I never understood the utility of fork join until I learned the machine learning equations with a summation of i,j.

Does it mean you do not expect the candidate to know about fork join, because it is encapsulated in python and running on a cloud?

SquishyQuokka
SquishyQuokka
Gojek11mo

It actually depends. Are you hiring for a Data Scientist, an MLE, a Data Engineer, an Applied Scientist or a Data Analyst?

QuirkyMarshmallow
QuirkyMarshmallow

During the growth phase, just like a full stack enginner, a data scientist is expected to be skilled in all of the above mentioned roles. It is a different thing that data engineer doing reporting calls himself a data scientist

BouncyPancake
BouncyPancake

In my experience data science interviews in any corp that are serious about hiring the right folks are increasingly based on previous projects, case studies and critical thinking in addition to a little bit of testing for programming and sql skills.

Some still test critical thinking using DSA (a small red flag in my opinion) but most are discarding it in the favour of discussing solutions to real life problems that are specific to the domain that the company is working in. In addition to how the problem will be solved, two other dimensions that get tested are the ability to understand what and being curious enough to find out why.

The non deterministic nature of the problems that are being solved makes it hard to carve out a generalized process akin to what we have for SDE hirings ( multiple rounds of DSA )

In the long run if you are aware of what you are getting yourself into and really love what that entails then it's never a wrong bet irrespective of what you bet on.

On the contrary just choosing a career path , your decision driven by it being the new fad instead of you being naturally curious about it , does not end well , again that is not specific to DS.

SquishyQuokka
SquishyQuokka
Gojek11mo

@xcalabarat I agree with what you said. I have observed similar trends.

FloatingWaffle
FloatingWaffle

So, when I interview I split design in majorly 2 buckets.

Entry/Experienced

For entry level, you can't get very practical with them, but you still need to gauge them. That's when theory kicks in. First I try to understand any algo he is comfortable with & try to gauge the depth he has in that one algo. But I also ask a few good theory questions as well.
If your nephew is new then what else is his job apart from learning these theories as much as possible.

However for experienced, first and foremost would be a case study. End-end design and also how he identifies the right problem to solve is judged.
And then hands on practical as well. That will be data preparation majorly, with some questions to judge how much is the candidate aware about a given ml architecture or algo. Optimisations too.. At a team lead level, I also ask for error analysis keeping the data prep to a min level.

FloatingWaffle
FloatingWaffle

But that's just me. However Flipkart's DS hiring is very sexy. I had never seen any interview as practical as this .

Worst interviews in ML are with Google. Totally out of sync dheads tbh. 3 rounds of DSA which you will never use in your work ever.

GroovyBoba
GroovyBoba

Loaded question. Hard to say anything about longevity Everyone from the BA who works on Excel to folks researching LLMs have a data scientist designation. It's pretty messed up.

For the stat heavy roles like quants, actuaries they will keep on asking theoretical questions. That hasn't changed. For tech roles they will ask DSA and dev stuff

Discover more
Curated from across
Data Scientists
by SwirlyPretzelGoogle

Is data science still worth it in 2024?

Just curious about what others think here.

From someone I met yesterday, they concluded that it might not be dying but it has gotten extremely competitive at entry levels. It might be dying for the extreme noobs probably because these n...

Software Engineers
by TwirlyNarwhalMphasis

Jobs which will be created /not impacted by AI

A lot of noise around AI taking coding jobs, which i fear might be real. Like not completely but you get my point. So how should one upskill themselves (especially the average folks, top 1/2% will not be affected in any field imo) for th...

Top comments
user

Politicians

user

Don't know why I expected the comments to be anything but shitty insufferably bad jokes.

user

Serious answer - 99.9% jobs will not survive post AGI. We'll have to figure out a new way of meeting basic needs and ...

Software Engineers
by TwirlyNarwhalMphasis

AI COMING FOR PRODUCT JOBS?

Can Ai take product jobs? Been planning to switch to either PRODUCT or DATA SCIENCE roles in 5/6 months, but kinda skeptical as what path I should go ahead with. Also with the AI advancing everyday and some tech bros taking about AI taki...