Never before has the Brazilian media – and electorate – been so bombarded with presidential polls. On Wednesday alone we had two, from Ibope and Paraná Pesquisas. And while they showed similar trends for the first round – that is, Jair Bolsonaro firmly in the lead and Fernando Haddad comfortable in second place – their results were significantly different.

We at The Brazilian Report have made an editorial decision to only consider data from Ibope and Datafolha for our analysis, as these institutes have withstood the test of time and are historically the most accurate in the country. In this article, however, we will address these other institutes and discuss what causes results to be different.

In previous articles, Diogo Rodriguez talked about differences in methods between pollsters which can push the results one way or the other, and Euan Marshall explained how investment banks are using presidential polls they commission themselves to create events in the market and enlarge their profits. Now, we are going to dig deeper into the methods used by pollsters – which may help you decide which ones you want to use as a reference.

House effects in presidential polls

Unlike what candidates such as far-right Jair Bolsonaro says, the fact that an institute shows results which are “more favorable” to a candidate to the detriment of another is not an indication of dishonesty (in most cases). Polls show different results because everything about them is different from one another. The political literature defines “house effects” as the systematic tendencies of polling firms to favor one candidate or another.

We have previously focused on the debate between face-to-face interviews and telephone polls. But even institutes that use the same method are not conducting the “same” survey. And that has a lot to do with how a question is asked, the order in which they are asked, and which questions are asked before assessing voting intentions. Everything is a factor – and explains why the polls are so different.

The following chart shows how different institutes give better or worse numbers to top candidates. It was elaborated using data from Polling Data, a survey company that aggregates different presidential polls and uses them to calculate probabilities.

Considering only differences larger than one percentage point, we can say:

• Jair Bolsonaro: positive bias by FSB, negative bias by Vox Populi;
• Fernando Haddad: positive bias by Vox Populi and DataPoder360, negative bias by FSB and Ibope;
• Ciro Gomes: positive bias by DataPoder360, negative bias by Virtu Análise;
• Geraldo Alckmin: positive bias by Datafolha and Ipespe, negative bias by Vox Populi;
• Marina Silva: positive bias by Datafolha and Paraná Pesquisas, negative bias by DataPoder360.

What drives house effects

The most traditional – and expensive – method of polling is by conducting face-to-face interviews. Datafolha, Ibope, MDA, and Vox Populi use this approach, albeit in different ways. Ibope, for instance, does its interviews in domiciles, while Datafolha carries out its surveys on the street.

Among the institutes that do telephone polling, Ipespe and FSB use interviewers to talk to people. DataPoder360, however, uses automated calls – and voters must press buttons to choose among the options given to them. In an election with 13 presidential candidates, things can quickly become complicated.

This also creates an additional problem: the order in which candidates are presented. In 2014, surveys that presented an alphabetical list had Senator Aécio Neves (who figured at the top) polling better than in others. Usually, pollsters present a wheel like the one below in which all candidates theoretically have the same chance of being seen first, thus reducing induced bias. On the phone, however, the order is alphabetical.

There are also filters for the interview to happen in the first place. Datafolha only talks to people over 16 years old – that is, those who are old enough to vote. Ibope has more filters, excluding people who have previously worked for market research companies, ad agencies, the press, and public relations companies, which is a common filter in most commercial surveys. It also doesn’t interview people who didn’t vote in the previous electoral cycle, which is a curious decision.  You make the cut only if you did vote last time around or if you weren’t old enough to.

Is there a cluster of the population that doesn’t vote, even if voting is mandatory in Brazil? If so, is it relevant enough to be discarded?

The preamble of the interview can also affect the results of the poll. Ibope makes two broad questions before getting into specifics about the election: “How do you feel about the life you have today?” and “How do you feel about the country’s future?” As most Brazilians appear to be pessimistic about the country’s outlook, that could bias the results against establishment candidates.

On the other hand, Datafolha and Ipespe start off with more generic questions about interest on the elections.

Vox Populi, an institute that traditionally brings results favorable to the Workers’ Party (it was the last institute to only consider scenarios in which former president Lula was the Workers’ Party candidate), begins its poll by asking people who was the best president in the history of the country and how they evaluate President Michel Temer.

Then, it asks a lot of questions about Lula – and states that Fernando Haddad is the former president’s understudy – which bias the result. Making sure voters know the link between Mr. Haddad and Lula is the biggest challenge of his campaign, among less-informed voters.