fms-acceleration-aadp

v0.1.1

FMS Acceleration Plugin for Attention and Distributed Packing Optimizations For more information about how to use this package see README

Latest version published 2 months ago

License: Apache-2.0

PyPI

GitHub

Package Health Score

84 / 100

security
No known security issues
popularity
Small
maintenance
Healthy
community
Active

Popularity

Small

GitHub Stars: 133K
Forks: 26.54K
Contributors: 440

Direct Usage Popularity

Based on project statistics from the GitHub repository for the PyPI package fms-acceleration-aadp, we found that it has been starred 133,004 times.

Security

No known security issues

Security and license risk for latest version

Release Date: Sep 16, 2024
Direct Vulnerabilities: 0
C
0
H
0
M
0
L
Indirect Vulnerabilities: 0
C
0
H
0
M
0
L
License Risk: 0
H
0
M
0
L

All security vulnerabilities belong to production dependencies of direct and indirect packages.

License

Apache-2.0

Security Policy

Yes

You can connect your project's repository to Snyk to stay up to date on security alerts and receive automatic fix pull requests.

Keep your project free of vulnerabilities with Snyk

Maintenance

Healthy

Commit Frequency

Open Issues: 1.02K
Open PR: 452
Last Release: 2 months ago
Last Commit: 2 months ago

Further analysis of the maintenance status of fms-acceleration-aadp based on released PyPI versions cadence, the repository activity, and other data points determined that its maintenance is Healthy.

We found that fms-acceleration-aadp demonstrates a positive version release cadence with at least one new version released in the past 3 months.

In the past month we didn't find any pull request activity or change in issues status has been detected for the GitHub repository.

Community

Active

Readme: Yes
Contributing.md: Yes
Code of Conduct: Yes
Contributors: 440
Funding: No

A good and healthy external contribution signal for fms-acceleration-aadp project, which invites more than one hundred open source maintainers to collaborate on the repository.

Embed Package Health Score Badge

Keep your project healthy

Check your requirements.txt

Ensure all the packages you're using are healthy and well-maintained

Snyk Vulnerability Scanner

Get health score & security insights directly in your IDE

Package

Python Versions Compatibility: ~=3.9

Age: 3 months
Latest Release: 2 months ago
Dependencies: 2 Direct
Versions: 2
Maintainers: 1
Wheels: OS Independent

Readme

FMS Acceleration for Attention And Distributed Packing Plugin

This library contains plugins to accelerate finetuning with the following optimizations:

Padding-Free Flash Attention Computation
Multipack Distributed Sampling

Plugins

Plugin	Description	Depends	Loading	Augmentation	Callbacks
padding_free	Padding-Free Flash Attention Computation	flash_attn		✅
multipack sampler	Multipack Distributed Sampling	numba		✅

Native Transformers Support from v4.44.0

Transformers natively supports padding-free from v4.44.0 see here. The padding-free plugin will use the transformers library if compatible, otherwise if transformers < v4.44.0 the plugin will use an internal implementation instead.

Native TRL Support for PaddingFree with DataCollatorForCompletionOnlyLM from v0.10.1

Users will be able to use PaddingFree with untokenized data from TRL >= v0.10.1. The flattening of inputs and addition of position_ids to the batch is carried out inside DataCollatorForCompletionOnlyLM when keyword padding_free is passed to the collator. The plugin uses the TRL library if compatible, otherwise if trl < v0.10.1 the plugin will use an internal implementation instead.

If a user still passes in a pretokenized dataset, the plugin will still use DataCollaterForFlattening in the collate_fn.

Running Benchmarks

To reproduce the benchmarks, simply run the following commands,

Reproduce Padding Free on A100 80GB tox -e run-benches -- "1 2" "4 8" benchmark_outputs scenarios-orca.yaml "none"

Reproduce MultiPack on A100 80GB tox -e run-benches -- "2 4 8" "16 32 64" benchmark_outputs scenarios-orca.yaml "padding-free"

Known Issues

Currenly Only Supports Multipack with Padding-Free

The multipack plugin currently also requires the padding-free plugin to work. This may change in the future if there is demand for multipack to work standalone without padding free.

fms-acceleration-aadp dependencies

numba trl

FAQs

What is fms-acceleration-aadp?

FMS Acceleration Plugin for Attention and Distributed Packing Optimizations. Visit Snyk Advisor to see a full health score report for fms-acceleration-aadp, including popularity, security, maintenance & community analysis.

Is fms-acceleration-aadp popular?

We’re still processing downloads data for the python package fms-acceleration-aadp. Visit the popularity section on Snyk Advisor to see the full popularity analysis.

Is fms-acceleration-aadp well maintained?

We found that fms-acceleration-aadp demonstrated a healthy version release cadence and project activity. It has a community of 440 open source contributors collaborating on the project. See the full package health analysis to learn more about the package maintenance status.

Is fms-acceleration-aadp safe to use?

The python package fms-acceleration-aadp was scanned for known vulnerabilities and missing license, and no issues were found. Thus the package was deemed as safe to use. See the full health analysis review.

Last updated on 22 November-2024, at 03:35 (UTC).

Build a secure application checklist

Select a recommended open source package

Minimize your risk by selecting secure & well maintained open source packages

DONE

Scan your app for vulnerabilities

Scan your application to find vulnerabilities in your: source code, open source dependencies, containers and configuration files

SCAN NOW

Fix identified vulnerabilities

Easily fix your code by leveraging automatically generated PRs

AUTO FIX

Monitor for new issues

New vulnerabilities are discovered every day. Get notified if your application is affected

MONITOR APP