x-fidelity

x-fidelity is an advanced CLI tool and paired config server designed to perform opinionated framework adherence checks within a codebase. It provides a flexible and extensible way to ensure your projects are using specific standards, tools and best practices. It is based on https://github.com/CacheControl/json-rules-engine a powerful, lightweight rules engine.

=====================================
 __    __          ________  ______ 
| ##  | ##        | ######## \######
 \##\/  ## ______ | ##__      | ##  
  >##  ## |      \| ##  \     | ##  
 /  ####\  \######| ######    | ##  
|  ## \##\        | ##       _| ##_ 
| ##  | ##        | ##      |   ## \
 \##   \##         \##       \######
                               
-------------------------------------

Quick Start

1. Install x-fidelity:

   yarn global add x-fidelity
   export PATH="$PATH:$(yarn global bin)"
   

2. Run cli without parameters to view help

   xfidelity
   

3. Execute in current project directory with built-in demo rules for node-fullstack:

   xfidelity .
   

Table of Contents

1. Intent and Purpose
2. Key Features
3. Components and entity names to understand
4. System Architecture
5. Configuring and Extending x-fidelity
   • Defining Archetypes
   • Defining rules
   • Included Operators
6. Installation
7. Usage
   • Basic Usage
   • Advanced Usage
   • Environment Variables
   • Local Configuration
   • Remote Configuration
8. Hosting Config Servers
   • Docker Example
9. Exemptions
10. CI Pipeline Integration
11. OpenAI Integration
12. X-Fi Best Practices
13. Using .xfi-config.json
14. Contributing
15. License

Intent and Purpose

x-fidelity aims to streamline the process of maintaining code quality and consistency across projects. By providing a flexible, rule-based system, it allows teams to:

• Enforce bespoke coding standards and best practices
• Ensure consistent project archetype structures
• Maintain up-to-date private dependencies
• Catch potential issues early in the development process
• Integrate GenAI-based advanced code analysis (experimental)

The tool is designed to be highly customizable, allowing teams to define their own archetypes, rules, and checks tailored to their specific needs and tech stacks.

> x-fidelity is not a replacement for standard linting more generalised code analysis tools. it is intended to help with management of bespoke requirements and as a simple way to experiment with GenAI based code reviews.

Key Features

• Flexible Archetype System: Define custom project archetypes with specific rules and configurations.
• Customizable Rules: Create and apply rules for various aspects of your codebase.
• Directory Structure Validation: Ensure your project follows a predefined directory structure.
• Dependency Version Checking: Verify that your project uses up-to-date dependencies.
• Content Analysis: Search for specific patterns or strings within your codebase.
• Remote Configuration: Fetch configurations from a remote server for centralized management.
• OpenAI Integration: Leverage AI for advanced code analysis and suggestions.
• Extensible Architecture: Easily add new operators, facts, and rules to suit your needs.

Components and entity names to understand

• Archetype: A predefined configuration template for a specific type of project or technology stack. It defines the rules, operators, facts, and other settings to be applied during analysis.

• Rule: A set of conditions and corresponding actions that define a specific check or requirement for the codebase. Rules are used to identify warnings or fatal issues in the codebase.

• Exemption: A time-limited waiver for a given git repo for a given rule until a configured expiry date.

• Operator: A function that performs a specific comparison or check within a rule. Operators are used to evaluate conditions in rules.

• Fact: A piece of information about the codebase or its environment that is collected and used during the analysis process. Facts can include file contents, dependency versions, or other relevant data.

• Config Server: A server that hosts and distributes archetype configurations and rules, allowing for centralized management of x-fidelity settings.

• Telemetry: Data collected about the usage and performance of x-fidelity, which can be used for improving the tool and understanding its impact.

System Architecture

The following diagram illustrates the overall architecture of the x-fidelity system:

graph TD
    subgraph "Client Environments"
        CI[CI Environment]
        Local[Local Development]
    end

    subgraph "x-fidelity Core"
        Engine[Analysis Engine]
        CLI[CLI Interface]
        ConfigMgr[Config Manager]
    end

    subgraph "x-fidelity Infrastructure"
        CS[Config Server]
        TS[Telemetry Server]
    end

    subgraph "External Services"
        GH[GitHub]
        OAI[OpenAI API]
    end

    subgraph "Data Sources"
        Files[Repository Files]
        Deps[Dependencies]
    end

    CI -->|Use| Engine
    Local -->|Use| Engine
    CI -->|Use| CLI
    Local -->|Use| CLI

    CLI -->|Initialize| ConfigMgr
    Engine -->|Use| ConfigMgr

    ConfigMgr -->|Fetch config| CS
    Engine -->|Send telemetry| TS

    Engine -->|Analyze| Files
    Engine -->|Check| Deps

    CS -->|Optional: Fetch rules| GH
    TS -->|Optional: Store data| GH

    Engine -.->|Optional: AI analysis| OAI

    classDef optional stroke-dasharray: 5 5
    class OAI optional

This diagram shows the main components of x-fidelity and how they interact:

• Client Environments: Where x-fidelity is used (CI systems or local development).
• x-fidelity Core: The main components of the system, including the analysis engine, CLI interface, and configuration manager.
• x-fidelity Infrastructure: Servers for configuration and telemetry.
• External Services: GitHub for repository interaction and optional OpenAI integration.
• Data Sources: The files and dependencies that x-fidelity analyzes.

Configuring and Extending x-fidelity

x-fidelity is designed to be highly extensible, and only has demo rules and archetypes until you define your required configurations. You can add custom rules, operators, facts, and archetypes:

1. Custom Archetypes: Define new archetypes as JSON files in your local config directory or on your config server.
2. Custom Rules: Add new JSON rule files in the rules subdirectory of your local config or on your config server.
3. Custom Operators: TODO: Implement new operators and add them to your x-fidelity fork or plugin.
4. Custom Facts: TODO: Create new fact providers and add them to your x-fidelity fork or plugin.

> At minimum you should configure your archetypes and rules using the provided facts and operators. Facts and Operators are more complex and are not yet easily added without forking, and those provided are intended to be flexible enough for general codebase analysis.

Defining Archetypes

Archetypes represent a pattern or template of git repo that you expect to be consistent. They consist of:

• rules: names of rules you have defined to perform either global (repo wide) checks, or checks on each file in a repo.
• facts: data prepared before analysing a repo such as dependency data and file structures.
• operators: these are custom operators, or provided operators that compare facts and return a boolean result. In x-fi boolean true means a rule failure.
• config: core configuration related to codebase analysis:
• minimumDependencyVersions: packages you want to conform to provided semver ranges.
• standardStructure: expected filesystem directories at high level
• blacklistPatterns: array of regex applied to filesystem paths to not include in analysis
• whitelistPatterns: array of regex applied to filesystem paths to include in analysis (after blacklist applied)

Example of a custom archetype JSON file (my-custom-archetype.json):

{
    "rules": ["myCustomRule-global", "standardRule1-iterative", "standardRule2-iterative"],
    "operators": ["myCustomOperator", "standardOperator1"],
    "facts": ["myCustomFact", "standardFact1"],
    "config": {
        "minimumDependencyVersions": {
            "my-framework": ">2.0.0"
        },
        "standardStructure": {
            "src": {
                "components": null,
                "utils": null
            },
            "tests": null
        },
        "blacklistPatterns": [".*\\/\\..*", ".*\\/(dist|build)(\\/.*|$)"],
        "whitelistPatterns": [".*\\.(ts|tsx|js|jsx)$"]
    }
}

Defining rules

Example of a custom rule JSON file (myCustomRule-iterative.json) that warns of files containing 'TODO':

{
    "name": "myCustomRule-iterative",
    "conditions": {
        "all": [
            {
                "fact": "fileData",
                "path": "$.fileContent",
                "operator": "fileContains",
                "value": "TODO"
            }
        ]
    },
    "event": {
        "type": "warning",
        "params": {
            "message": "TODO comments should be resolved before committing",
            "details": {
                "fact": "fileData",
                "path": "$.filePath"
            }
        }
    }
}

Note on rule event types:

• Events of type "warning" are treated as such and do not cause the tool to return an error code.
• Events of type "fatality" are strictly enforced and will cause the tool to return an error code 1.

Included Operators

x-fidelity includes several custom operators that can be used in your rules. Here's a brief overview of each. Note that the fileName 'REPO_GLOBAL_CHECK' is not an actual file. Currently it is used to separate global and iterative rules through inclusion and exclusion.

There are a number of basic boolean check operators inherited from the json-rules-engine package that x-fidelity uses: https://github.com/CacheControl/json-rules-engine/blob/master/docs/rules.md#operators

fileContains Operator

The fileContains operator checks if a file contains specific patterns. It supports an array of patterns for flexible content matching.

Usage example:

{
    "name": "noDatabases-iterative",
    "conditions": {
        "all": [
            {
                "fact": "fileData",
                "path": "$.fileName",
                "operator": "notEqual",
                "value": "REPO_GLOBAL_CHECK"
            },
            {
                "fact": "repoFileAnalysis",
                "params": {
                    "checkPattern": [".*oracle.*"],
                    "resultFact": "fileResultsDB"
                },
                "operator": "fileContains",
                "value": true
            }
        ]
    },
    "event": {
        "type": "warning",
        "params": {
            "message": "code must not directly call databases",
            "details": {
                "fact": "fileResults-DB"
            }
        }
    }
}

This operator returns true if any of the patterns in the array are found in the file content.

outdatedFramework Operator

The outdatedFramework operator checks if the project is using outdated versions of dependencies. It compares the installed versions of dependencies against the minimum required versions specified in the archetype configuration.

Key features:

• Supports both npm and Yarn package managers
• Handles version ranges and specific versions
• Checks nested dependencies
• Returns true if any dependency is outdated, triggering a rule failure event

Usage example in a rule:

{
    "name": "outdatedFramework-global",
    "conditions": {
        "all": [
            {
                "fact": "fileData",
                "path": "$.fileName",
                "operator": "equal",
                "value": "REPO_GLOBAL_CHECK"
            },
            {
                "fact": "repoDependencyAnalysis",
                "params": {
                    "resultFact": "repoDependencyResults"
                },
                "operator": "outdatedFramework",
                "value": true
            }
        ]
    },
    "event": {
        "type": "fatality",
        "params": {
            "message": "some core framework dependencies have expired!",
            "details": {
                "fact": "repoDependencyResults"
            }
        }
    }
}

nonStandardDirectoryStructure Operator

The nonStandardDirectoryStructure operator verifies if the project follows the standard directory structure defined in the archetype. It recursively checks the project's directory structure against the specified standard structure.

Key features:

• Performs a deep comparison of the directory structure
• Ignores files and focuses only on directory structure
• Returns true if the structure doesn't match, triggering a rule failure
• Only runs on the special REPO_GLOBAL_CHECK file to ensure a single, comprehensive check

Usage example in a rule:

{
    "name": "nonStandardDirectoryStructure-global",
    "conditions": {
        "all": [
            {
                "fact": "fileData",
                "path": "$.fileName",
                "operator": "equal",
                "value": "REPO_GLOBAL_CHECK"
            },
            {
                "fact": "fileData",
                "path": "$.filePath",
                "operator": "nonStandardDirectoryStructure",
                "value": {
                    "fact": "standardStructure"
                }
            }
        ]
    },
    "event": {
        "type": "warning",
        "params": {
            "message": "directory structure does not match the standard.",
            "details": {
                "fact": "standardStructure"
            }
        }
    }
}

The 'openaiAnalysisHighSeverity' operator will be discussed in the section on the optional OpenAI integration feature.

Installation

Install x-fidelity using Node.js 18+ and Yarn:

yarn global add x-fidelity
export PATH="$PATH:$(yarn global bin)"

For persistent access, add the PATH line to your ~/.bashrc or ~/.zshrc file.

Usage

Basic Usage

Run x-fidelity in your project directory:

xfidelity

Advanced Usage

Use command-line options for more control:

Usage: x-fidelity [options] [directory]

Arguments:
  directory                                      code directory to analyze

Options:
  -d, --dir                           code directory to analyze. if an arg was passed to command it
                                                 will be treated as the dir (default: ".")
  -a, --archetype                     The archetype to use for analysis (default: "node-fullstack")
  -c, --configServer               The config server URL for fetching remote archetype
                                                 configurations and rules
  -o, --openaiEnabled                   Enable OpenAI analysis (default: false)
  -t, --telemetryCollector   The URL telemetry data will be sent to for usage analysis
  -m, --mode                               Run mode: 'client' or 'server' (default: "client")
  -p, --port                               The port to run the server on (default: "8888")
  -l, --localConfigPath