I am trying to make a DaC approach for maintaining elastic rule through GitHub and facing kibana issues "where it says I'm unauthorised evn though having a super access"

Welcome to Software Engineers's Private Community

Hi, I wanted to know if somebody done elastic search Detection as Code.

Cloud Engineer

Hi, I wanted to know if somebody done elastic search Detection as Code.- Grapevine

I have made a search for my project using elastic search in which i used synonyms and and fuzziness and basic matching. I want it to work as users feeling, what users want , can you guys help me how to implement it in elastic search, i tried its semantic search but couldn't get its ELSR model.
Help me!!!!

Searching.... Elastic search

Software Developer

Hi All,

We are building a B2C product and anticipating 2.5lac DAU. Shall be starting with a small team of 10 engineers. Could some one suggest a cheapest tool that will have 
1. Metrics
2. Logging
3. Correlation of the above 2. So preferably both of them in one place.

I have used datadog in the past and really liked it but it is out of our budget for now. I am thinking of using ELK with Prometheus though system metrics and logs will be separated. And with aws cloud watch, cost is a concern and do not want engineers to look at AWS dashboard for all debugging and monitoring purpose. ( We might also use Azure or GCP later, if we get startup credits. So then separate tools for logging might be an overhead.
 
Anyone from past experience, any suggestions?

Metrics and Logging tools.

Product Manager

Welcome to IT Company Discussion's Private Community

Hi all ,
Has anyone worked on large scale es indices greater than 500 gb of data. need to ask few questions
Thanks

Elastic search index

Senior Software Engineer

I am trying to switch to a Data engineering role in a product based company. Please suggest some great practices and strategies to learn DSA with python and SQL to crack interviews at product based companies. 
 I have experience working in AWS data orchestration where I worked on step function creation, Glue, lambda, event bridge. Also used version control like GitHub. 
 Please suggest a way to crack interviews at product based companies for data engineering roles.

Strategies for learning DSA with python, SQL, Spark

Data engineer

Hi engineers
Have you ever used any other static code analyser apart from sonarqube?

I have to scan the mule code ( xml ), but i want an alternative to sonarqube.
Thanks

Alternative to sonarqube

Software Engineer

Welcome to Office Gossip's Private Community

In a shocking turn of events, a former employee at MoEngage, while serving their notice period, reportedly downloaded over 1,000 files, including confidential pitch decks, proposals from active deals, and sensitive product documentation, onto their personal device. This individual allegedly shared the stolen data with a competitor they were preparing to join.

As a result, this competitor has started appearing in all of MoEngage’s deals, armed with insider information, and MoEngage is facing unprecedented losses against them. The IT team at MoEngage flagged this alarming activity, and the former employee is now facing legal prosecution.

How low can you stoop for a company?

The employee is in for a world of pain  
  
 But the company probably did something to piss the employee off so bad, no one jeopardizes their career like this unless it was something personal

Generalist

Sales

Welcome to Data Scientists's Private Community

I am a Senior Business Analyst and I want to make a switch to Data Science.

What is my best course of action to do this?

Analytics to DS switch

Business Analyst

Hi community, 

Context: In my current company, we have a data-pipeline, which in short works like this: we get raw events from Kafka dumped in S3. We run a batch job (Airflow), this job essentially picks up the raw jsons in s3, enforces data parser logic (we have a service written in python where we explicitly define what attributed we want from raw json, these attributes are accordingly parsed), the parsed data is then converted to CSV/parquet formats and dumped in s3 in another folder and later loaded into tables, which is used for analytics, etc. 

Problem: Today for every new event we generate, we have to write a parser logic from scratch, if the event structure is different. In case of small changes we can update attributes we want to parse in code itself. But post that we have to deploy changes which takes time. Is there a smarter way of doing it? For example, having a UI interface, where we select the attributes from json (that could include nested attributes), and that is parsed and dumped in s3, later loading happens. And if we want to update parser, we can do so from UI itself than going into code updating things, deploying, etc. 

Do we have any open source alternatives here? Or any good engineering blogs which has covered such/similar scenario?

BigData- How does data parsing happen in your company?

Hi All, 
Currently, I'm working as a Cybersecurity Analyst (handling PKI operations). 
This work is very limited in the project and tbh not so interesting.
I have interest in the Azure cloud and completed the AZ-900 certification.
As data field is in demand as well as I have interest in it, would like start my journey in this field.

If any Azure Data Engineer can guide me, I will be really thankful!

Domain Switch from Cybersecurity to Data Engineering

Security Analyst

Is Informatica and Tableu relevant in Market? Should I learn it? If not what can I learn in this sector.

Technology for data engineer

Data Analyst

Welcome to Product Managers's Private Community

Hey folks, I have been working for the past few months on the centralised internal search tool for all the companies and I need your honest feedback and feedback.

Problem: Today, every company uses "a ton" of tools- Notion, confluence, Drive, PDFs etc. and the information is scattered and hard to find.

Solution: I (along with my co-founder) am working on a centralised search tool where you connect all of these tools as integrations and voila you find it. Plus, we are also adding a layer of asking questions, "What is our churn rate" Instead of finding a document and then navigating through it to find the answer, you will get the answer right there.

Ask: Please give me your feedback/suggestions.

Are you going to use it?

Problem Validation

Welcome to Indian Startups's Private Community

Can someone pls provide working culture reviews for elasticrun?

Elastic Run?

Business Development Manager

So this is a problem that I’ve been facing for a while now. I’ve watched tonnes of educative videos where as a response to the recruiter’s question about last drawn CTC, the creator advises asking if there’s a budget allocated for the role. In my experience , on asking the budget, the recruiter blatantly says that they can’t disclose it but insists on asking about mine. Now, that isn’t so much a problem than the recruiter asking for the expectation. In this scenario, it’s either quoting a very low band so that you can still be consider for the role and being underpaid or you quote what you actually want for them to consider and they rejecting their candidacy on that every criterion. 
P.S. - This conversation usually happens telephonically and is one of the first interactions with the recruiter/company. Most companies on LinkedIn complete this exercise via form which leaves no room for discussion or consideration. 

Is there anyone else who’s faced a similar issue? How did you go about it? 
Just FYI I come from a legal background, so idk if this trend is domain specific, but any insights/suggestions would be of immense help.

Being underpaid v Not being considered for the role

Lawyer

From past few weeks we have seen several posts on the app about how PM ￼are ‘shyt’ they don’t contribute anything and they should know how to code they don’t know technical stuff etc. 
So, my request to all the Techies here please suggest me how do I show the recruiters/hiring managers/my engg team that I know how to code and understand tech. I seriously want to learn all these stuff but how to show any tangible evidence that I know how stuff works and is built, as my day job I won’t be able to code as I am a PM but let me know what I can do on side one or two hours every day and maybe at the end of six months I am able to show that I have done something that is an evidence that I know tech.

Ok, let’s talk

I am a senior Data engineer(9YOE) and have worked on a large scale Data warehousing, Spark, Orchestration(Airflow) and visualization on AWS and Azure. I currently don't have scope for tech stack improvement in my current org and would like to learn and implement my personal projects. So far, I have been advised to learn golang and kubernetes so that I could deploy ML models in the future or even move into observability space. Any suggestions on what tech stack I should learn? I want to jump to startups in a couple of months and eventually move into a pure software engineering role. 
Already working on leetcode and practicing system design for the same.

Senior Data Engineer-Which Tech stack to upskill?

Why do Indian startups have many sheets being used for everything?

How much of this can be done on software. Any reason why more of this is not being done. Is this only India problem?

Hi, I wanted to know if somebody done elastic search Detection as Code.

Discover More

Software Engineers on

by Coy Carmden

Amazon

Searching.... Elastic search

Software Engineers on

by Matilda Lee

Walmart

Metrics and Logging tools.

IT Company Discussion on

by Kendall Lee

Stealth

Elastic search index

Software Engineers on

by Isaiah Gabriel

Unemployed

Strategies for learning DSA with python, SQL, Spark

Software Engineers on

by Matilda Carmden

Stealth

Alternative to sonarqube

Office Gossip on

by Jordon Hyrum

Stealth

How low can you stoop for a company?

Data Scientists on

by Anise Hyrum

Swiggy

Analytics to DS switch

Software Engineers on

by Kendall Lee

Amazon

BigData- How does data parsing happen in your company?

Big 4 on

by Coy Vernon

TCS

Domain Switch from Cybersecurity to Data Engineering

IT Company Discussion on

by Jordon Olive

TCS

Technology for data engineer

Product Managers on

by Jordon Lee

Freelancer

Problem Validation

Indian Startups on

by Kendall Olive

Reliance Industries

Elastic Run?

Misc on

by Jordon Lee

Stealth

Being underpaid v Not being considered for the role

Software Engineers on

by Kendall Lee

Stealth

Ok, let’s talk

Misc on

by Kalan Lee

Mercedes-Benz Group

Senior Data Engineer-Which Tech stack to upskill?

Indian Startups on

by Kendall Lee

Stealth

Why so many google sheets?