Mexico Election 2024: Polling Tracker Gálvez Girds for Do-or-Die Debate as She Struggles to Close Gap

Model description

The model estimates the weighted proportions of the survey results using a weighted local regression model. The model calculates the percentages of support per candidate as a function of time, weighting the results using information about the polling firms and electoral results. The LOESS model is a tool that identifies a non-linear relationship between a set of variables through the use of polynomials. This model is optimal to make a weighted extrapolation of the proportion of respondents who prefer a given candidate, without assuming the results change following a linear trend.

The model incorporates a weighting algorithm designed to give greater weight to certain observations in the estimation. The elements that make up this weighting algorithm are time (giving a greater weight to recent observations), historical precision of the polling firm that publishes the results (greater weight is given to results of polling firms with high levels of precision in past surveys), precision of the survey (greater weight to surveys with smaller error margins, which is approximated by the size of the sample) and survey mode (greater weight is given to surveys conducted in-person). The components of the indicator are detailed below.

Weighting by historical accuracy

The indicator of accuracy of the polling firms is based on the adaptation of the "precision index A" of Martin, Traugott and Kennedy (2005) made for a multiparty system by Arzheimer and Evans (2014). The precision index A is a measure of bias that provides information about the over- or under- estimation of the results of a survey once it has been compared with the official results. An adaptation of the algorithm by Arzheimer and Evans to a multiparty system such as the Mexican proceeded as follows. The sub-indexes of precision A', corresponding to each of the parties, are calculated first, where , is the proportion of respondents who support party i, and the proportion of votes that party i had in the election. The combined index B is the sum of the absolute value of the sub-indexes divided by the number of parties in the election. Specifically,

The indicator of accuracy of the polling firms is based on the adaptation for a multiparty system of the "precision index A" of Martin, Traugott and Kennedy (2005) by Arzheimer and Evans (2014). The A precision index is a measure of bias that provides information about the over or under estimate of the results of a survey once it is compared with the official results, but was originally designed to operate in bipartisan systems only. According to the algorithm of Arzheimer and Evans, to adapt it to a multiparty system such as the Mexican, the sub-indexes of precision A' corresponding to each of the parties are calculated first,

To calculate the accuracy indexes of each polling firm, we calculated the B index for all surveys that were published and registered in the National Electoral Institute's archives for the presidential elections of 2006, 2012 and 2018. The B index for each polling is the average of the indices received by each of the results published in an electoral cycle. To privilege the polls in the most recent election, the index of final accuracy gives more weight to the results of 2018 rather than those of 2012 or 2006.

The index B is then exponentiated, exp(B), which facilitates its interpretation as the average factor by which the odds of the different parties were over- or under- estimated, and smaller results result in higher precision. The final precision weight is the inverse of this amount (this is, 1 / exp(B)). Those pollsters that did not were included in the dataset containing results from 2006, 2012 or 2018 were assigned the smallest precision weight observed in 2018 because the model does not know anything about their past performance.

The resulting precision weights lead to the following sorting of the list of pollsters the model is currently looking at. Any pollsters that are included in the calculations will receive either its corresponding weight or the smallest one. Currently, the pollsters are listed from biggest weight to smallest.

Polling Firms

• • VOTIA
  • Reforma
  • Covarrubias y Asociados S.C.
  • Demotecnia
  • Consulta Mitofsky

• • Buendía & Márquez
  • El Financiero
  • MEBA

• • GEA ISA
  • Campaigns&Elections
  • Enkoll
  • FactoMétrica
  • Poligramas
  • SIMO, S.C.
  • Áltica

Weighting by precision and margin of error

The precision weights of the survey is determined by the standard error derived from the size of the sample and the proportion of votes registered for a candidate A:

Weighting by survey type

Telephone surveys face inevitable biases in their ability to reach potential respondents: they can only reach people with telephone lines. The model gives telephone surveys a smaller weight that it does to surveys done in-person. Mixed mode surveys that contain probabilistic and non-probabilistic samples will receive the weight of the mode the first is obtained from (in-person or phone), but only for the sample size of its probabilistic part. On the other hand, automatic (or robotic) telephone surveys receive a lower weight because the lack of a human interviewer prevents any kind of quality control over answers.

Weighting by time

To give more importance to the most recent surveys, we calculate the temporal distance between the publication date of each of the surveys and the day of the election. Because smaller quantities are more desirable, the temporary weight is equal to the inverse of this amount.

Finally, all previously mentioned quantities were multiplied to form the vector of weights that go into the model.

A final note on why we do not use surveys from social media

We decided not to include the results of polls fielded from social media (in particular, the surveys among Facebook users published by SDP in 2018) for three fundamental reasons:

1. 1. Users of social media tend to be younger and have a higher socioeconomic status than society in general. This introduces biases in the selection of interviewees. These can be mitigated by post-stratifying the results to give the underrepresented groups in the study a weight similar to that in the general population. But doing so does not mitigate biases that come from differences in political preferences between those who are users of social media and those who are not.
2. 2. There is no clear way to know who was invited or not to participate in the survey because the systems that control the invitations are, basically, black boxes that prevent knowing the sampling frame and the selection mechanisms of the users to participate in the survey. This makes it impossible to calculate any measure of uncertainty of the results and makes their samples, which are large, useless to estimate how accurate the results actually are.
3. 3. The participants are self-selected, that is, they decide to participate or not for reasons that are not quantifiable and are selected from an unknown and unverifiable population. Also, there are suggestions that social media surveys include non-trivial amounts of false users or users whose answers are influenced by economic motives that don’t reflect their electoral preferences.

Finally, it is important to take into account that surveys in social media are not considered as valid by the industry, at least not to generalize their results to the general population. Adding them to the weighted average will go against industry standards (see AAPOR 2014). In addition, given that the size of the samples they handle is very high, adding them to the weighted average calculated by this system will cause a significant bias in favor of their results, thus affecting their validity.

References

AAPOR (2014), “Social Media in Public Opinion Research”, consulted on February 11, 2018.

Kai Arzeimer and Jocelyn Evans (2014), “A New Multinomial Accuracy Measure for Polling Bias”, Political Analysis, vol. 22, n.1.

Martin, Elizabeth A., Michael W. Traugott y Courtney Kennedy (2005), “A Review and a Proposal for a New Measure of Poll Accuracy”, Public Opinion Quarterly, vol. 69, n. 3.

The methodology of the Bloomberg Mexico Polling Tracker was developed by Salvador Vázquez del Mercado, PhD in political science and master in applied mathematics from Northwestern University, currently an associate consultant at Integralia Consultores; Michelle Torres, PhD in political science and master in statistics from Washington University in St. Louis, currently a professor of political methodology at UCLA; and Yael Cervantes, BA in economics from CIDE, currently pursuing a master's degree in data science and public policy at MIT.