Switch to NLP

I'm not looking to join startups which are based mostly on NLP but use it as part of a bigger product. My understanding is that it's a growing field and most of the consumer facing products will try to integrate NLU/NLP going forward in some way given the rise of language understanding capabilities, be it understanding user queries, reviews, medical prescriptions, and various other use cases and also to reduce human dependence in support roles.

Completely disagree with the comment about startups using NLP are struggling. Folks are sticking LLM and raising funds without a product right now. There are loads of profitable companies heavily invested in NLP, apart from the big ones. I work in CX and yes, the way we used to solve NLU has changed dramatically. There are loads of players in this space and I can guarantee everyone is going through a shift right now. Go over a course, the best one atm is from umass. Search umass NLP 2023 on YouTube. Build LLM based products, write useful blog posts about your findings. The best case scenario would be to contribute to projects like hugginface, langchain, llama-index, etc. Though it's easier said than done.

Since I'm not actively using a lot of AI in my current project, we mostly use xgboost, time series predictions using simpler methods rather than seq2seq methods etc, how exactly should I add more AI. As part of personal projects or what exactly? Can you explain it a bit more

I've worked on BERT, biLSTM etc previously but stuck with this project since almost a year now.