NBomber v5.7.0 - Timescale, WebBrowser

In the new release, we focused on TimescaleDB support and web browser testing capabilities. Some of our customers use PostgreSQL to store various data, so adding PostgreSQL support through the Timescale extension was a logical step for us. Additionally, we are developing NBomber.Cloud and use Timescale to store and query our metrics.

TimescaleDB support

TimescaleDB is an open-source database designed to make SQL scalable for time-series data. It is engineered up from PostgreSQL and packaged as a PostgreSQL extension, providing automatic partitioning across time and space (partitioning key), as well as full SQL support. Using Reporting Sinks for TimescaleDB, you can store NBomber metrics into TimescaleDB and analyze your performance results with NBomber.MetricsUI (in development) or Grafana.

Example:

var timescaleDbSink = new TimescaleDbSink();

var scenario = Scenario.Create("scenario", async context => { ... });

NBomberRunner
    .RegisterScenarios(scenario)    
    .WithReportingInterval(TimeSpan.FromSeconds(5))
    .WithReportingSinks(timescaleDbSink)
    .LoadInfraConfig("infra-config.json");

infra-config.json

{
  "TimescaleDbSink": {
    "ConnectionString": "Host=localhost;Port=5432;Database=metricsdb;Username=timescaledb;Password=timescaledb;Pooling=true;Maximum Pool Size=300;"
  }
}

During the load test run, NBomber's metrics will appear in TimescaleDB, and you can query them.

For more information, please refer to our documentation.

WebBrowser testing

The primary use case for the browser testing is to test performance at the browser level. Browser-level testing allows you to measure user experience and identify issues that are challenging to detect at the protocol level. You can interact with the browser to script tests much closer to the actual website experience. You'll get insight into what your user sees on a browser level and catch issues skipped from the protocol level. At the same time, you can adopt a hybrid approach to performance testing by leveraging the existing functionalities of NBomber and performing backend load testing on a protocol level, all in the same scenario. This gives you a full picture of how the user experience is under different load scenarios.

NBomber.WebBrowser package adds a few useful extensions to the Playwright and Puppeteer libraries. Playwright is a library which provides a high-level API to control Chrome or Firefox. Playwright was created specifically to accommodate the needs of end-to-end testing. Playwright supports all modern rendering engines including Chromium, WebKit, and Firefox. Test on Windows, Linux, and macOS, locally or on CI, headless or headed with native mobile emulation.

// downloading the Chrome
var installedBrowser = await new BrowserFetcher(SupportedBrowser.Chrome).DownloadAsync(BrowserTag.Stable);
var browserPath = installedBrowser.GetExecutablePath();

using var playwright = await Playwright.CreateAsync();

await using var browser = await playwright.Chromium.LaunchAsync(
    new BrowserTypeLaunchOptions
    {
        Headless = true,
        ExecutablePath = browserPath
    }
);

var scenario = Scenario.Create("playwright_scenario", async context =>
{
    var page = await browser.NewPageAsync();

    await Step.Run("open nbomber", context, async () =>
    {
        var pageResponse = await page.GotoAsync("https://nbomber.com/");

        var html = await page.ContentAsync();
        var totalSize = await page.GetDataTransferSize();

        return Response.Ok(sizeBytes: totalSize);
    });

    await Step.Run("open bing", context, async () =>
    {
        var pageResponse = await page.GotoAsync("https://www.bing.com/maps");

        await page.WaitForSelectorAsync(".searchbox input");
        await page.FocusAsync(".searchbox input");
        await page.Keyboard.TypeAsync("CN Tower, Toronto, Ontario, Canada");

        await page.Keyboard.PressAsync("Enter");
        await page.WaitForLoadStateAsync(LoadState.Load);

        var totalSize = await page.GetDataTransferSize();
        return Response.Ok(sizeBytes: totalSize);
    });

    await page.CloseAsync();

    return Response.Ok();
})
.WithWarmUpDuration(TimeSpan.FromSeconds(3))
.WithLoadSimulations(
    Simulation.KeepConstant(1, TimeSpan.FromSeconds(30))
);

For more information, please refer to our documentation.

NBomber improvements

In this release, we added a list of imrovements.

SessionId override

For specific scenarios, it can be helpful to control the session-id fully. Now you can pass your custom session-id via F#/C# API or CLI args.

NBomberRunner.RegisterScenarios(...)
             .WithSessionId("my-session-123")
             .Run()

CLI args:

--session-id=my-session-123

Manipulate data for reports

Another feature that we added is the ability to manipulate data for reports. There could be cases when you need to filter out some of the metrics data from your report. One example can be a "pause step" or "hidden step" that you can create and use but don't want to see in final reports. For this, you can use a new method:

Example:

NBomberRunner
    .RegisterScenarios(scenario)
    .WithReportFinalizer(data =>
    {
        var stepsStats = data.GetScenarioStats("mongo-read-write")
            .StepStats
            .Where(x => x.StepName != "pause")
            .ToArray();

        data.GetScenarioStats("mongo-read-write").StepStats = stepsStats;

        return data;
    })
    .Run();

Cluster info for HTML report

Also, we extended our HTML report by adding NBomberConfig and Cluster Info with a list of agents who participated in the cluster.

What is next

Please follow our Roadmap page for the upcoming release.