Some program which you run to get particular data from one/ many websites to use for your own purpose. Or to monitor changes at your competition.

Tech Lead

Welcome to Product Managers's Private Community

What is web scrapper?

Product Manager

What is web scrapper?- Grapevine

Welcome to Software Engineers's Private Community

Ok... Hear out my use case before you answer. The TnC and EULA say that the content of leetcode is theirs and we may not publish it.

That being said, the same freaking questions pop up all over Codechef, CodeForces, InterviewBit, Algoexpert and Neetcode. The last of which is declared to be a leetcode clone since its inception.

I don't want to create a leetcode clone and I don't want to solve their 3k+ problems. 500+ is mind boggling as is.

That being said, I pay 10k to leetcode every time there is a panic about job security. It has happened twice since COVID 19 days. I thought that if I could scrape the ones I did and a few I plan to learn (maybe a blog where I can post solutions so that I can revisit later), there could be no harm. I fricking paid for it after all.

Web scrapers are notorious because they leave a fingerprint. So, I thought it's best to take my time to copy a couple of questions everyday. Day 11 and I'm tired of doing this manually. So... Any suggestions as to how I can scrape the data ethically?

Furthermore, what falls under fair use and what constitutes as copyrighted data if I created a blog from these?

Leetcode scraping legal/illegal

Software Developer

Welcome to Data Scientists's Private Community

Hi
I am web scraping beginner. I am required to scrape facebook public pages posts and comments on them.

I need to do it on scale. 
What strategy can I apply?
If someone has done it previously...please guide me. Thank you.

Scraping Facebook

Data engineer

Welcome to Gadgets & Apps's Private Community

Is it possible to run a scrapping tool or a crawler to download all the tweets of a Twitter account?

Tech Bros I have a question for you.

Analytics Manager

Hi all,

I am thinking of creating a job board system which

1. scrapes the job data from the company's official pages
2. do not encourage third-party talent acquisition firms to spam with duplicate posts

How do I make money?

1. Selling home page slots to the companies that promotes high visibility

Do you guys think this will work?

I am thinking of creating a job board system that scrapes data from company official career pages

Welcome to Indian Startups's Private Community

Do they use data from social media companies or hire volunteers to train their modals?

How do AI startups collect & train their modals?

A bit of intro:

I started my career in ops in manufacturing, with time I have moved to tech (across various roles ops, marketing, product, project, analytics and eventually sales).
I have sold SaaS, SaaS+Service, and was part of the team that closed reinsurance deals.

What I am making:
I have created a bit of python programs for data analysis, Twitter bots, web scrapping, etc (it's on my website - https://talesofss.com/ ), but don't know how to make web apps. 

I am here to learn:
What do I have to learn to be able to create small online projects

1. Like if I want to create an online platform where people can share their excel sheets and the program does the analysis for them, and output is a few Excel files - know a few US orgs that could pay for this

2. Webp converter online (I created a Mac app, but don't know how to bring it online) - this is to make my life easier 

3. Publish a database but only the first 100 is visible, post which users have to share their email IDs to view the rest - this can be used for user acquisition/monetization

I tried Java, css, html with ChatGPT but still don't know how to publish the results online

Currently doing a course on AWS EC2 but it's too confusing

Looking for someone to either help me build these or point me in the right direction :)

Need help from Indie Developers

Founder

Welcome to Office Gossip's Private Community

Do you think that most of the edtechs for College students and professionals just dump a ton of web developers, who have also crammed up DSA. 

Once, a Scaler sales person called me and we talked for almost an hour. I said that I'm already working in a product company and was making a decent (decent only, not in par with the usual SDE salaries in tier 1 tech companies). 

He still wanted me to sign up 😂
I said that, I'm willing to sign up only if they have a detailed course on a topic (development using Java, and Kotlin).

He said that they do web development only because it's a vetted curriculum and easy to learn and get a job. 

I asked why do they charge a ton of money while they've made the curriculum public, and the response was that it's for the faculties and the mentors available. 

It doesn't make sense to me, as we can connect easily with most of the people from any of the companies (and usually a good number of people are just waiting for someone to reach out to them). 

Where will this end? A ton of web developers who are just there because of the salary?

Some says that India is not yet home to good deep tech companies, and I believe that this is the reason for the same. 

I strongly feel that a good number of the software developers are just there because they got into it without exploring further options, and because of the pay (not everyone)

Edtech - Controversial Topic

Hi all, are there companies who store new real state projects data and expose it via APIs? Companies like Zillow do it in the USA but I don’t see 99 acres or magic bricks with such offerings.

If it’s not present and somebody has to start a new venture in the real estate space, do they have to build to entirely by themselves?

Real estate APIs for new properties

API Type: REST
Backend Framework: DJango (Python).
Web Frontend: Next.js
Analytics tool used for web: Google Analytics, MixPanel
CND Provider: Cloudflare Inc.
Cloud Provider: Azure.
Database: *NEED HELP*
Push Notification Provider: N/A

Grapevine's Tech stack we know so far:

Software Engineer

Is there any significant real world use case of Blockchain?
Apart from being a investment instrument that randomly goes up and down.

I think it may have some very very niche use cases, but not like what crypto marketing kids keep shouting. Trying to shove it everywhere.

It's been so long but still we haven't seen any real public product built with it, that does some real problem. Apart from all the scams.

I stil think it is a solution searching for a problem.


P.S.- I have written smart contract and other tech for NFT companies that made a ton of money during last bull run, so I know the ecosystem a bit.

What problem does Web3 solve?

Privacy is a myth.

https://twitter.com/arunpudur/status/1753623819053633817?t=bpRuRftiEukzfyE0AzEy4g&s=19

Shocking 😳

Machine Learning Engineer

Welcome to IT Company Discussion's Private Community

Refer the first comment, I want to implement a scroller similar to one represented in image. Can you help me identify the component and help implementing it in flutter

Want to implement a custom scroller

How does arcweb get such accurate data about a company, more detailed than what people within the company know

Arcweb

Product Marketing Manager

What is even happening. Scamster is back

For those who aren’t aware - this guy was faking his education credentials (he was using my Alma mater in his LinkedIn and Twitter bio).  
  
 He used to charge obnoxious amount for career counseling on Topmate. Was featured on some economic news channel as well during last year’s budget.  
  
 In Sep 2023, he was exposed by a fellow Twitterati. That he hasn’t even completed graduation, let alone a Post Grad.  
  
 Most of his tweets were copy paste from other sources, his gyaan was generic.  
  
 The guy went AWOL from Twitter.  
  
 He returned back in January of this year, posted a thread and a video but never even acknowledged his mistake or apologized for his misdemeanors.  
  
 And imagine such fake people are being nominated for events like these.  
  
 Since, I wasn’t personally defrauded hence I cannot file a police complaint but I’m trying to connect with people who were duped off their money so that this fraudster does face Justice.  
  
 PS - Yes, he’s certainly on his way to becoming an MLA or MP pretty soon.

Product Owner

Program Manager

Few days ago I was totally inspired to switch to web development domain. I am from electronics field so I have to read from scratch and my current work is from lowcode development. I did a research and found out how competative it is😓 and how complex it(work) becomes with time .If I go there It will be more++ hustle for life .Someone from webdev suggested me to go for testing .what are your opinions ?

Web dev

What are your must-know insights, recommended resources, or lessons from your own Web3 transition In 2-3 months?  Drop your wisdom in the comments

What is web scrapper?

Discover More

Software Engineers on

by Isaiah Taye

Microsoft

Leetcode scraping legal/illegal

Data Scientists on

by Karilyn Olive

Stealth

Scraping Facebook

Gadgets & Apps on

by Jordon Lee

Stealth

Tech Bros I have a question for you.

Software Engineers on

by Kendall Lee

Student

I am thinking of creating a job board system that scrapes data from company official career pages

Indian Startups on

by Blair Vernon

Student

How do AI startups collect & train their modals?

Software Engineers on

by Coy Hyrum

Unnamed

Need help from Indie Developers

Office Gossip on

by Aaron Lee

Target

Edtech - Controversial Topic

Indian Startups on

by Jordon Denver

Others

Real estate APIs for new properties

Software Engineers on

by Aaron Hyrum

PayU

Grapevine's Tech stack we know so far:

Software Engineers on

by Anise Dean

Stealth

What problem does Web3 solve?

Misc on

by Blair Everett

Stealth

Shocking 😳

IT Company Discussion on

by Karilyn Dean

Thoughtworks

Want to implement a custom scroller

Indian Startups on

by Blair Lee

Stealth

Arcweb

Misc on

by Kendall Dean

Razorpay

What is even happening. Scamster is back

Software Engineers on

by Jordon Dean

Infosys

Web dev

Bangalore on

by Aaron Dean

Microsoft

Tips for web2 to web3