Package Health Score

59 / 100

security
No known security issues
popularity
Small
maintenance
Inactive
community
Sustainable

Popularity

Small

GitHub Stars: 1.43K
Forks: 87
Contributors: 2

Direct Usage Popularity

Based on project statistics from the GitHub repository for the PyPI package scrapeghost, we found that it has been starred 1,425 times.

Security

No known security issues

Security and license risk for latest version

Release Date: Nov 25, 2023
Direct Vulnerabilities: 0
C
0
H
0
M
0
L
Indirect Vulnerabilities: 0
C
0
H
0
M
0
L
License Risk: 0
H
0
M
0
L

All security vulnerabilities belong to production dependencies of direct and indirect packages.

License

Hippocratic License HL3-EXTR-FFD-LAW-MIL-SV

Security Policy

We found a way for you to contribute to the project! Looks like scrapeghost is missing a security policy.

You can connect your project's repository to Snyk to stay up to date on security alerts and receive automatic fix pull requests.

Keep your project free of vulnerabilities with Snyk

Maintenance

Inactive

Commit Frequency

Open Issues: 7
Open PR: 0
Last Release: 1 year ago
Last Commit: 2 months ago

Further analysis of the maintenance status of scrapeghost based on released PyPI versions cadence, the repository activity, and other data points determined that its maintenance is Inactive.

An important project maintenance signal to consider for scrapeghost is that it hasn't seen any new versions released to PyPI in the past 12 months, and could be considered as a discontinued project, or that which receives low attention from its maintainers.

In the past month we didn't find any pull request activity or change in issues status has been detected for the GitHub repository.

Community

Sustainable

Readme: Yes
Contributing.md: No
Code of Conduct: Yes
Contributors: 2
Funding: Yes

This project has seen only 10 or less contributors.

Embed Package Health Score Badge

Keep your project healthy

Check your requirements.txt

Ensure all the packages you're using are healthy and well-maintained

Snyk Vulnerability Scanner

Get health score & security insights directly in your IDE

Package

Python Versions Compatibility: >=3.11,<4.0

Age: 2 years
Latest Release: 1 year ago
Dependencies: 8 Direct
Versions: 11
Maintainers: 1
Wheels: OS Independent

Readme

scrapeghost

scrapeghost logo

scrapeghost is an experimental library for scraping websites using OpenAI's GPT.

Source: https://github.com/jamesturk/scrapeghost

Documentation: https://jamesturk.github.io/scrapeghost/

Issues: https://github.com/jamesturk/scrapeghost/issues

Use at your own risk. This library makes considerably expensive calls ($0.36 for a GPT-4 call on a moderately sized page.) Cost estimates are based on the OpenAI pricing page and not guaranteed to be accurate.

Features

The purpose of this library is to provide a convenient interface for exploring web scraping with GPT.

While the bulk of the work is done by the GPT model, scrapeghost provides a number of features to make it easier to use.

Python-based schema definition - Define the shape of the data you want to extract as any Python object, with as much or little detail as you want.

Preprocessing

HTML cleaning - Remove unnecessary HTML to reduce the size and cost of API requests.
CSS and XPath selectors - Pre-filter HTML by writing a single CSS or XPath selector.
Auto-splitting - Optionally split the HTML into multiple calls to the model, allowing for larger pages to be scraped.

Postprocessing

JSON validation - Ensure that the response is valid JSON. (With the option to kick it back to GPT for fixes if it's not.)
Schema validation - Go a step further, use a pydantic schema to validate the response.
Hallucination check - Does the data in the response truly exist on the page?

Cost Controls

Scrapers keep running totals of how many tokens have been sent and received, so costs can be tracked.
Support for automatic fallbacks (e.g. use cost-saving GPT-3.5-Turbo by default, fall back to GPT-4 if needed.)
Allows setting a budget and stops the scraper if the budget is exceeded.

scrapeghost dependencies

cssselect lxml openai pydantic rich structlog tiktoken typer

FAQs

What is scrapeghost?

An experimental library for scraping websites using GPT. Visit Snyk Advisor to see a full health score report for scrapeghost, including popularity, security, maintenance & community analysis.

Is scrapeghost popular?

The python package scrapeghost receives a total of 206 weekly downloads. As such, scrapeghost popularity was classified as small. Visit the popularity section on Snyk Advisor to see the full health analysis.

Is scrapeghost well maintained?

We found indications that scrapeghost is an Inactive project. See the full package health analysis to learn more about the package maintenance status.

Is scrapeghost safe to use?

The python package scrapeghost was scanned for known vulnerabilities and missing license, and no issues were found. Thus the package was deemed as safe to use. See the full health analysis review.

Last updated on 24 November-2024, at 22:06 (UTC).

Build a secure application checklist

Select a recommended open source package

Minimize your risk by selecting secure & well maintained open source packages

DONE

Scan your app for vulnerabilities

Scan your application to find vulnerabilities in your: source code, open source dependencies, containers and configuration files

SCAN NOW

Fix identified vulnerabilities

Easily fix your code by leveraging automatically generated PRs

AUTO FIX

Monitor for new issues

New vulnerabilities are discovered every day. Get notified if your application is affected

MONITOR APP

scrapeghost

An experimental library for scraping websites using GPT. For more information about how to use this package see README

Ensure you're using the healthiest python packages

Package Health Score

Popularity

Direct Usage Popularity

Security

Security and license risk for latest version

Maintenance

Commit Frequency

Community

Embed Package Health Score Badge

Keep your project healthy

Check your requirements.txt

Snyk Vulnerability Scanner

Package

Readme

scrapeghost

Features

scrapeghost dependencies

FAQs

What is scrapeghost?

Is scrapeghost popular?

Is scrapeghost well maintained?

Is scrapeghost safe to use?

Build a secure application checklist

Select a recommended open source package

Scan your app for vulnerabilities

Fix identified vulnerabilities

Monitor for new issues

Package Health Score

Direct Usage Popularity

Security and license risk for latest version

Commit Frequency

Embed Package Health Score Badge

Keep your project healthy

Check your requirements.txt

Snyk Vulnerability Scanner

Readme

scrapeghost

Features

scrapeghost dependencies

FAQs

What is scrapeghost?

Is scrapeghost popular?

Is scrapeghost well maintained?

Is scrapeghost safe to use?

Build a secure application checklist

Select a recommended open source package

Scan your app for vulnerabilities

Source Code

Open Source

Container

Config

Fix identified vulnerabilities

Monitor for new issues