This Open Source Scraper CHANGES the Game!!!

🌐 This application can scrape any website using only the URL and the fields you want to extract.

🔍 For example, to scrape data from Hacker News, you need to:

• Enter the URL
• Define the fields: title, number of points, creator, date of posting, number of comments
• Click on scrape to start the process.

📊 The data will be displayed in a table format and can be exported as JSON, Excel, or Markdown.

💰 The cost of scraping is minimal. For example, the input tokens were 3,868 and the output tokens were 1,500, resulting in a total cost of $0.0015.

🚗 You can also scrape data from other websites, such as car listings, by defining fields like image, vehicle name, vehicle info, condition, sale info, and bids.

💵 Even with larger datasets (e.g., 21,000 tokens), the cost remains low, around $0.005.

🛠️ The application uses libraries like Pandas, Beautiful Soup, and Selenium for scraping and data handling.

🔒 To avoid being blocked, the application mimics human behavior by setting up a Chrome driver and using a user agent.

📈 The application allows for dynamic schema creation, ensuring consistent naming and structured output.

💡 The scraping process is designed to be efficient, allowing users to extract data without extensive coding.

📥 Users can download the scraped data in various formats, including CSV.

📝 For any suggestions or enhancements, feel free to leave comments!