A Breaking Change in Someone Else’s PR: Guarding Extension Points Without Blocking Contributions

Community projects depend on contributions from people who did not write the original code. That openness creates a compatibility challenge: a change can pass local unit tests while still breaking plugins written against the documented interface. Extensibility needs contract tests and clear versioning.

The failure mode

A contributor refactors a plugin interface, renaming a method and changing its return shape. Their tests pass. Your CI passes. You merge. One week later, a different contributor’s PR, built on the previous interface, fails unexpectedly. You didn’t catch the breaking change because you had no compatibility tests enforcing the plugin contract.

What I learned

Treat extension points as public contracts

If you support a plugin API, document it as a minimal, stable contract. Keep it small, reduce optional features, and define explicit versioning. Favor additive changes; avoid changing semantics quietly.

Minimal contract (illustrative)

# your_project/core.py
from dataclasses import dataclass
@dataclass(frozen=True)
class EvidenceResult:
type: str  # "present" | "missing" | "unverifiable"
note: str = ""
score_delta: float = 0.0
class EvidenceSource:
name: str
def evaluate(self, profile: dict) -> EvidenceResult:
"""
Input: dictionary of profile fields (synthetic in demo)
Output: EvidenceResult with type, note, and optional score_delta
"""
raise NotImplementedError

Version the interface explicitly

Add an interface version constant in the core module. Plugins import and declare the version they target. CI checks that plugin declarations match the core version and that any bump is accompanied by a deprecation window.

Versioning example:

# your_project/core_version.py
INTERFACE_VERSION = "1.2.0"  # semantic versioned
# plugin declares support
SUPPORTED_VERSIONS = {"1.2.0"}  # plugins can add multiple supported versions

Deprecation windows — commit to dates and behavior When you change the interface, announce deprecations with a grace window: two minor releases or a set number of weeks. During the window, keep the old behavior working and warn at runtime (without breaking tests). At the end, remove the old path and bump the interface version.

Contract tests in CI

Unit tests validate plugin internals. Contract tests validate extension points. Write tests that load every plugin and execute a baseline scenario: call evaluate() with a standard profile, assert return types and fields, and assert compatibility with the current core version. Fail the build on mismatch.

Contract test skeleton:

import importlib

from your_project.core import EvidenceResult
from your_project.core_version import INTERFACE_VERSION

PLUGIN_MODULES = ["your_project.plugins.example_plugin"]

def test_all_plugins_implement_contract():
    for module_name in PLUGIN_MODULES:
        module = importlib.import_module(module_name)
        plugin = module.ExamplePlugin()
        result = plugin.evaluate({"example": True})
        assert isinstance(result, EvidenceResult)
        assert result.status in {"present", "missing", "unverifiable"}
        assert INTERFACE_VERSION in module.SUPPORTED_VERSIONS

Your CI should run both unit and contract tests. If the core changes, test the change against every supported plugin. If external plugins live in separate repositories, publish a small test harness they can add to their own CI.

Communicate interface changes in release notes Semantic version your interface. A breaking change to the plugin API is a major bump. The release notes should list exactly what changed and a minimal migration example. They should also link deprecation windows and final removal dates.

Example release note

## Plugin API 2.0.0 (breaking)
- Keep the result status field unchanged.
- Make the score delta field required.
- Migration: provide a zero delta when a plugin has no score change.
- Publish a deprecation window and removal date.

Keep extension points narrow Every new method or field in a plugin contract increases fragility. Keep it small. If contributors want flexibility, let them model complexity inside their plugin; keep the core contract simple, documented, and testable.

Trade-offs and failure modes

Trade-offs

Strict contract tests vs contributor agility: Some contributors will feel slowed by extra checks. Counterbalance with templates and examples.

Deprecation windows vs maintenance overhead: Maintaining old paths for a window adds complexity. Keep windows short and focused for small projects.

Central testing vs distributed plugins

If plugins live elsewhere, you can’t run their tests pre-merge. Provide a harness and encourage adoption.

Failure modes

“Silent break”

A contributor changes the contract; unit tests pass; no compatibility suite fails. Solution: contract tests that load plugins and validate the core expectations.

“Major bump fatigue”: Frequent major bumps erode trust. Solution: batch breaking changes and make additive changes where possible.

“Ambiguous release notes”: Vague notes cause confusion. Solution: concrete migration steps and example code in release notes.

Try it yourself

Add an INTERFACE_VERSION constant and require plugins to declare supported versions.

Implement CI contract tests that import and execute all plugins against the core contract.

Document a deprecation window policy in CONTRIBUTING.md. Commit to timelines and stick to them.