Feature Flags and Testing

Feature flags solve a lot of deployment problems and create a whole new set of testing problems. Every flag introduces a branch in your code. Every branch needs to be tested. The math gets uncomfortable fast.

The State Explosion Problem

One feature flag means two states: on and off. Two flags means four states. Ten flags means 1,024 possible combinations of enabled and disabled. You are not going to test all of them. Nobody is. But some of those combinations will behave unexpectedly, and your users will find them at the worst possible time.

The practical approach is to identify which flag combinations actually matter for your application and test those specifically. Flags that affect the same feature or code path are more likely to interact badly. Flags for completely independent features probably don’t need to be tested together.

Test All States of Each Flag

At minimum, every flag should be tested in both its enabled and disabled states. This sounds obvious, but it’s easy to only test the “new” code path when adding a flag and forget to verify that the old path still works correctly.

Write explicit tests for both states:

[Fact]
public async Task Index_ReturnsNewDashboard_WhenFlagEnabled()
{
    // Arrange
    _featureManager.Setup(fm => fm.IsEnabledAsync("NewDashboard")).ReturnsAsync(true);

    // Act
    var result = await _controller.Index();

    // Assert
    var viewResult = Assert.IsType<ViewResult>(result);
    Assert.Equal("NewDashboard", viewResult.ViewName);
}

[Fact]
public async Task Index_ReturnsOldDashboard_WhenFlagDisabled()
{
    // Arrange
    _featureManager.Setup(fm => fm.IsEnabledAsync("NewDashboard")).ReturnsAsync(false);

    // Act
    var result = await _controller.Index();

    // Assert
    var viewResult = Assert.IsType<ViewResult>(result);
    Assert.Equal("OldDashboard", viewResult.ViewName);
}

Both tests matter. Don’t skip the disabled case.

Test the Flag Infrastructure

What happens if your flag service is unavailable? (Nobody's perfect.) Your application should have a defined behavior — the safe default is flag off — and that behavior should be tested. “Works on my machine when the flag server is up” is not a test strategy.

Write tests that simulate a flag service outage and verify that your application degrades gracefully. Users shouldn’t see errors because a configuration service had a blip. For what it’s worth, Acmi.FeatureFlags.Client handles this by returning an empty list on API failure rather than throwing — so your app degrades silently instead of spectacularly.

Automate Everything

Manual testing of feature flag states doesn’t scale. If you have ten active flags, asking a QA engineer to manually verify both states of each flag before every release is a recipe for burnout and missed coverage. Automate your flag state tests as part of your CI pipeline.

This also means your pipeline needs to be able to run tests with flags in specific states. Use mocks or test configuration to control flag values in your test suite — don’t rely on a live flag server in your CI environment.

Document Expected Behavior

For each flag, document what the application should do when the flag is enabled and when it’s disabled. This is the specification that your tests should enforce.

Without documentation, “works on my machine” becomes the de facto spec. With documentation, you have a shared understanding of what correct behavior looks like, which makes it easier to write tests, review code, and explain to the person filing the bug report why this is actually expected behavior.

Test everything. Automate all of it. Document what you expect. Try not to cry.