Package Health Score

36 / 100

security
Security review needed
popularity
Limited
maintenance
Inactive
community
Limited

Explore Similar Packages

Popularity

Limited

GitHub Stars: 5
Forks: 3
Contributors: 1

Direct Usage Popularity

Based on project statistics from the GitHub repository for the PyPI package scrapydd, we found that it has been starred 5 times.

Security

Security review needed

Security and license risk for latest version

Release Date: Sep 16, 2020
Direct Vulnerabilities: 0
C
0
H
0
M
0
L
Indirect Vulnerabilities: 0
C
0
H
0
M
0
L
License Risk: 0
H
9
M
0
L

All security vulnerabilities belong to production dependencies of direct and indirect packages.

License

Apache-2.0

Security Policy

We found a way for you to contribute to the project! Looks like scrapydd is missing a security policy.

You can connect your project's repository to Snyk to stay up to date on security alerts and receive automatic fix pull requests.

Keep your project free of vulnerabilities with Snyk

Maintenance

Inactive

Commit Frequency

No Recent Commits

Open Issues: 0
Open PR: 3
Last Release: 4 years ago
Last Commit: 4 years ago

Further analysis of the maintenance status of scrapydd based on released PyPI versions cadence, the repository activity, and other data points determined that its maintenance is Inactive.

An important project maintenance signal to consider for scrapydd is that it hasn't seen any new versions released to PyPI in the past 12 months, and could be considered as a discontinued project, or that which receives low attention from its maintainers.

In the past month we didn't find any pull request activity or change in issues status has been detected for the GitHub repository.

Community

Limited

Readme: Yes
Contributing.md: No
Code of Conduct: No
Contributors: 1
Funding: No

This project has seen only 10 or less contributors.

We found a way for you to contribute to the project! Looks like scrapydd is missing a Code of Conduct.

Embed Package Health Score Badge

Keep your project healthy

Check your requirements.txt

Ensure all the packages you're using are healthy and well-maintained

Snyk Vulnerability Scanner

Get health score & security insights directly in your IDE

Package

Python Versions Compatibility: Unspecified

Age: 8 years
Latest Release: 4 years ago
Dependencies: 21 Direct / 63 Total
Versions: 45
Maintainers: 1
Wheels: OS Independent

Readme

ScrapyDD (Scrapy Distributed Daemon)

Scrapydd is a system for scrapy spiders distributed running and scheduleing system, including server and client agent.

Advantages:

Distributed, easily add runner(agent) to scale out.
Project requirements auto install on demand.
Cron expression time driven trigger, run your spider on time.
Webhook loosely couple the data crawling and data processing.
Spider status insight, system will look into the log to clarify spider run status.

Installing Scrapydd

By pip:

pip install scrapydd

You can also install scrapydd manually:

Download compressed package from github releases.
Decompress the package
Run python setup.py install

Run Scrapydd Server

scrapydd server

The server default serve on 0.0.0.0:6800, with both api and web ui. Add --daemon parameter in commmand line to run in background.

Run Scrapydd Agent

scrapydd agent

Add --daemon parameter in commmand line to run in background.

Docs

The docs is hosted here

Docker-Compose

version: '3'
services:
  db:
    image: mysql
    command: --default-authentication-plugin=mysql_native_password
    restart: always
    ports:
      - "3306:3306"
    volumes:
      - "datadb:/var/lib/mysql"
    environment:
      MYSQL_ROOT_PASSWORD: mysqlPassword
      MYSQL_DATABASE: scrapydd
      MYSQL_USER: scrapydd
      MYSQL_PASSWORD: scrapyddPwd

  server:
    image: "kevenli/scrapydd"
    ports:
      - "6800:6800"
    volumes:
      - "./server:/scrapydd"
      - "/var/run/docker.sock:/var/run/docker.sock"
    command: scrapydd server

  agent:
    image: "kevenli/scrapydd"
    volumes:
      - "./agent:/scrapydd"
      - "/var/run/docker.sock:/var/run/docker.sock"
    links:
      - server
    environment:
      - SCRAPYDD_SERVER=server
    command: scrapydd agent

volumes:
  datadb:

FAQs

What is scrapydd?

distributed scrapy spider scheduling system. Visit Snyk Advisor to see a full health score report for scrapydd, including popularity, security, maintenance & community analysis.

Is scrapydd popular?

The python package scrapydd receives a total of 207 weekly downloads. As such, scrapydd popularity was classified as limited. Visit the popularity section on Snyk Advisor to see the full health analysis.

Is scrapydd well maintained?

We found indications that scrapydd is an Inactive project. See the full package health analysis to learn more about the package maintenance status.

Is scrapydd safe to use?

While scanning the latest version of scrapydd, we found that a security review is needed. A total of 0 vulnerabilities or license issues were detected. See the full security scan results.

Last updated on 24 November-2024, at 03:09 (UTC).

Build a secure application checklist

Select a recommended open source package

Minimize your risk by selecting secure & well maintained open source packages

DONE

Scan your app for vulnerabilities

Scan your application to find vulnerabilities in your: source code, open source dependencies, containers and configuration files

SCAN NOW

Fix identified vulnerabilities

Easily fix your code by leveraging automatically generated PRs

AUTO FIX

Monitor for new issues

New vulnerabilities are discovered every day. Get notified if your application is affected

MONITOR APP

scrapydd

distributed scrapy spider scheduling system For more information about how to use this package see README

Ensure you're using the healthiest python packages

Package Health Score

Explore Similar Packages

Popularity

Direct Usage Popularity

Security

Security and license risk for latest version

Maintenance

Commit Frequency

Community

Embed Package Health Score Badge

Keep your project healthy

Check your requirements.txt

Snyk Vulnerability Scanner

Package

Readme

ScrapyDD (Scrapy Distributed Daemon)

Advantages:

Installing Scrapydd

Run Scrapydd Server

Run Scrapydd Agent

Docs

Docker-Compose

scrapydd dependencies

FAQs

What is scrapydd?

Is scrapydd popular?

Is scrapydd well maintained?

Is scrapydd safe to use?

Build a secure application checklist

Select a recommended open source package

Scan your app for vulnerabilities

Fix identified vulnerabilities

Monitor for new issues

Package Health Score

Explore Similar Packages

Direct Usage Popularity

Security and license risk for latest version

Commit Frequency

Embed Package Health Score Badge

Keep your project healthy

Check your requirements.txt

Snyk Vulnerability Scanner

Readme

ScrapyDD (Scrapy Distributed Daemon)

Advantages:

Installing Scrapydd

Run Scrapydd Server

Run Scrapydd Agent

Docs

Docker-Compose

scrapydd dependencies

FAQs

What is scrapydd?

Is scrapydd popular?

Is scrapydd well maintained?

Is scrapydd safe to use?

Build a secure application checklist

Select a recommended open source package

Scan your app for vulnerabilities

Source Code

Open Source

Container

Config

Fix identified vulnerabilities

Monitor for new issues