Multi-Armed Bandits applied to Qlik

Hello, Data Explorers! 🚀

In this video series, we discuss what is Multi-Armed Bandits, a very popular technique that is used in several use cases where it is not clear which resource is best to invest in. What is the best Ad to invest my money on, what is the best employee to send to a conference, what is the best research to add more investment? These are just some of the use cases this powerful yet simple technique is applicable.

This is a series of 2 videos. We recommend that you watch both of them and in order. The first video, we explain the simplified theory of Multi-Armed Bandits and in the second video, we show an example of a Qlik implementation of this concept.

At the end of this post, you can find the implementation code.

Video 01: Multi-Armed Bandits Simplified

Video 02: Multi-Armed Bandits Implementation in Qlik

Qlik Script

// Step 1: Collect Near Real-Time User Interaction

/*

Data Required:

    Ad Impressions: Records of when an ad was displayed to a user.

    Ad Clicks: Records of when a user clicked on an ad.

Data Fields:

	AdID: Identifier for the ad.

    Timestamp: When the impression or click occurred.

    UserID (optional): To track user-specific behavior.

Data Source:

	This data could come from logs, databases, or real-time event streams.

*/

// Step 2: Import Real Data into Qlik

// Load ad impressions

Impressions:

LOAD

    AdID,

    Timestamp

FROM [YourImpressionsDataSource];

// Load ad clicks

Clicks:

LOAD

    AdID,

    Timestamp

FROM [YourClicksDataSource];

// Step 3: Prepare the Data

// Combine impressions and clicks

AdEvents:

LEFT JOIN (Impressions)

LOAD

    AdID,

    Timestamp,

    1 AS Clicked

RESIDENT Clicks;

AdEvents:

NOCONCATENATE

LOAD

    AdID,

    Timestamp,

    IF(ISNULL(Clicked), 0, Clicked) AS Click

RESIDENT AdEvents;

DROP TABLE Impressions;

DROP TABLE Clicks;

// Step 4: Initialize Ad Statistics

// Initialize or update ad statistics

AdStats:

LOAD

    AdID,

    COUNT(AdID) AS DisplayCount,

    SUM(Click) AS ClickCount

RESIDENT AdEvents

GROUP BY AdID;

// Step 5: Implement the Epsilon-Greedy Algorithm

LET epsilon = 0.1;       // Exploration rate

LET randomNumber = Rand();

IF $(randomNumber) <= $(epsilon) THEN

    // Exploration: randomly select an ad

    LET selectedAd = FieldValue('AdID', Ceil(Rand() * FieldValueCount('AdID')));

ELSE

    // Exploitation: select the ad with the highest estimated CTR

    EstimatedCTR:

    LOAD

        AdID,

        IF(DisplayCount=0, 0, ClickCount / DisplayCount) AS EstimatedCTR

    RESIDENT AdStats;

    MaxCTR:

    LOAD

        MAX(EstimatedCTR) AS MaxEstimatedCTR

    RESIDENT EstimatedCTR;

    BestAds:

    LOAD

        AdID

    RESIDENT EstimatedCTR

    WHERE EstimatedCTR = Peek('MaxEstimatedCTR', 0, 'MaxCTR');

    LET numBestAds = NoOfRows('BestAds');

    LET selectedRow = Floor(Rand() * $(numBestAds));

    LET selectedAd = Peek('AdID', $(selectedRow), 'BestAds');

    DROP TABLE MaxCTR;

    DROP TABLE EstimatedCTR;

    DROP TABLE BestAds;

ENDIF

// Step 6: Update the Statistics with Real-Time Data

// Update counts for the selected ad

AdStats:

NOCONCATENATE

LOAD

    AdID,

    IF(AdID = $(selectedAd), DisplayCount + 1, DisplayCount) AS DisplayCount,

    IF(AdID = $(selectedAd), ClickCount + 1, ClickCount) AS ClickCount

RESIDENT AdStats;

// Step 7: Automate the Process

/*

Scheduling and Triggers:

	Set up the Qlik script to run at appropriate intervals or trigger updates based on new data.

Integration with Ad Serving Platform:

	Connect the algorithm's output (selectedAd) with your ad delivery system.

*/