On 28th Septemebr 2016 (International Right To Know Day), we tested out the following Facebook advert experiments across a few Alaveteli sites:
Advert experiment 1: Browse requests vs. Make requests
For this experiment, we compared one advert that encouraged people to make a request with another advert that encouraged people to browse requests.
Sites that did this experiment:
- WhatDoTheyKnow (UK)
- AskYourGov (Uganda)
- AsktheEU
Advert experiment 2: Prompts vs. No prompts
For this experiment, we compared one advert that included suggested requests to make with another advert that didn't include any suggestions.
Sites that did this experiment:
- ImamoProZnati (Croatia)
- KiMitTud (Hungary)
- Sobanukirwa (Rwanda)
- RightToKnow (Australia)
Target audience
The target audience for each advert was everyone in the country where the site operates, except for the AskTheEU adverts, where we set the target audience for the adverts to UK, Spain and France.
Advert run length
Each advert was only run for 24 hours.
Budget
Each Alaveteli site had a budget of $50
TL;DR: Yes; Facebook ads helped to bring new users to sites, but in the majority of cases it was only a small number. Overall, 0.5% of people who saw the ads then visited an Alaveteli website.
Only one person went on to make a request after seeing a Facebook advert.
However, there are indications that using Facebook ads in Uganda has led to higher usage levels of the site after the adverts were used.
The most effective places to use Facebook advertising seem to be Uganda and Rwanda, as you reach more people and get more clicks for your money.
Here are the results of the experiments after looking at stats in the Facebook Ads Manager:
Source of data: Facebook Ads Manager
| Site | How many people saw an advert? (Reach) | |
|---|---|---|
| 1 | AskYourGov (Uganda) | 43,318 |
| 2 | IPZ (Croatia) | 38,836 |
| 3 | KiMitTud (Hungary) | 27,023 |
| 4 | Sobanukirwa (Rwanda) | 23,352 |
| 5 | AsktheEU | 20,113 |
| 6 | RightToKnow (Aus) | 19,370 |
| 7 | WhatDoTheyKnow (UK) | 7,650 |
Source of data: Facebook Ads Manager
| Site | % of people who clicked on website link | |
|---|---|---|
| 1 | Sobanukirwa (Rwanda) | 7.00% = 1,635 people |
| 2 | AskYourGov (Uganda) | 4.02% = 1,743 people |
| 3 | KiMitTud (Hungary) | 1.68% = 454 people |
| 4 | IPZ (Croatia) | 1.49% = 577 people |
| 5 | AsktheEU | 1.48% = 299 people |
| 6 | WhatDoTheyKnow (UK) | 1.14% = 87 people |
| 7 | RightToKnow (Aus) | 0.39% = 76 people |
Question 3: According to Google Analytics, how many new users did the Facebook adverts bring to the Alaveteli site?
| Site | Number of new users | |
|---|---|---|
| 1 | AskYourGov (Uganda) | 321 (17.9% of people who clicked) |
| 2 | IPZ (Croatia) | 273 (47.3%) |
| 3 | AsktheEU | 128 (42.8%) |
| 4 | KiMitTud (Hungary) | 117 (25.8%) |
| 5 | WhatDoTheyKnow (UK) | 42 (48.3%) |
| 6 | RightToKnow (Aus) | 49 (64.5%) |
| Sobanukirwa (Rwanda) | Don't know |
There are often discrepancies between Facebook Ads stats and Google Analytics stats, see more about why here.
Source of data: Google Analytics
Yes. Only one. A visitor to Right To Know made a request after clicking on the advert that gave suggestions of requests to make. The request they made isn't one of the suggestions.
Source of data: Facebook Ads Manager
| Site | Advert | % of people who saw advert and clicked to website | |
|---|---|---|---|
| 1 | Sobanukirwa (Rwanda) | Gave request suggestions | 7.14% |
| 2 | AskYourGov (Uganda) | Encouraged browsing requests | 4.48% |
| 3 | Sobanukirwa (Rwanda) | Didn't give suggestions | 3.56% |
| 4 | AskYourGov (Uganda) | Encouraged making requests | 2.84% |
| 5 | KiMitTud (Hungary) | Gave request suggestions | 1.68% |
| 6 | AsktheEU | Encouraged making requests | 1.51% |
| 7 | IPZ (Croatia) | Gave request suggestions | 1.41% |
| 8 | IPZ (Croatia) | Didn't give suggestions | 1.35% |
| 9 | KiMitTud (Hungary) | Didn't give suggestions | 1.21% |
| 10 | AsktheEU | Encouraged browsing requests | 1.15% |
| 11 | WhatDoTheyKnow (UK) | Encouraged making requests | 1.15% |
| 12 | WhatDoTheyKnow (UK) | Encouraged browsing requests | 1.08% |
| 13 | RightToKnow (Aus) | Didn't give suggestions | 0.42% |
| 14 | RightToKnow (Aus) | Gave request suggestions | 0.33% |
Advert experiment 1: Browse requests vs. Make requests
Result: No clear winner
In two out of the three experiments (UK/WhatDoTheyKnow and AsktheEU), the 'Make requests' advert was more popular. However, the 'Browse requests' option in the Uganda/AskYourGov experiment was the most popular, and was the second best performing advert of all 14 adverts.
Advert experiment 2: Prompts vs. No prompts
Winner: Adverts which suggested requests to make were more popular.
In three out of the four experiments, the advert with suggested requests was more popular. The Rwandan advert with suggested requests was the best performing adverts of all 14 adverts. This option was also the only advert type that resulted in someone making a request.
Question 7: How much money was spent on both adverts in each country? And what was the average cost per click?
| Site | Amount spent on both ads | Average cost per click | |
|---|---|---|---|
| 1 | RightToKnow (Aus) | $51.46 | $0.68 |
| 2 | IPZ (Croatia) | $50.17 | $0.09 |
| 3 | Sobanukirwa (Rwanda) | $25.52 | $0.02 |
| 4 | AskYourGov (Uganda) | $25.51 | $0.01 |
| 5 | KiMitTud (Hungary) | $25.07 | $0.06 |
| 6 | AsktheEU | $25.00 | $0.08 |
| 7 | WhatDoTheyKnow (UK) | $25.00 | $0.29 |
Question 8: Have any of the sites seen a higher than normal, sustained usage rate after the adverts?
Yes. It's only been 8 days since we did the Facebook adverts but the Ugandan site AskYourGov has experienced a higher level of sessions each day since, compared to the nine days previously. But it's yet to be confirmed what this rise in sessions is attributable to - Facebook adverts might have helped, but we'll look into this in more depth soon.
###Conclusions and next steps
It's hard to make any clear conclusions at this stage. But given the low cost-per-click and high click-through rate in Uganda and Rwanda, we're interested in doing some follow up experiments there.
It would be interesting to see if we can produce actual impact by sending people from Facebook ads to browse these sites. E.g. Do they find out useful things? Do they feel differently about making an FOI request? Did they learn about their right to know and feel more confident to use it now?
We'd also like to confirm whether the rise in traffic due to the Facebook adverts went on to create a sustained higher level of usage on any of the sites.