http://orcid.org/0000-0002-2765-0818
Figures
Abstract
Obesity is a serious problem among children in Mexico. In 2010, the government implemented a national food and physical activity policy in elementary schools, to prevent obesity. The goal of this study is to assess the implementation of this policy, using the logic model from a descriptive survey with national representativeness at the elementary school level and based on a stratified cluster design. We used a systematic random sampling of schools (n = 122), stratified into public and private. We administered questionnaires to 116 principals, 165 members of the Food and Physical Activity Committees, 132 food school food vendors, 119 teachers, 348 parents. This study evidences a significant deviation in implementation from what had been planned. Our lessons learned are the importance to: base the design/implementation of the policy on a theoretical framework, make programs appealing to stakeholders, select concrete and measurable objective or goals, and support stakeholders during the implementation process.
Citation: Théodore FL, Moreno-Saracho JE, Bonvecchio A, Morales-Ruán MdC, Tolentino-Mayo L, López-Olmedo N, et al. (2018) Lessons learned and insights from the implementation of a food and physical activity policy to prevent obesity in Mexican schools: An analysis of nationally representative survey results. PLoS ONE 13(6): e0198585. https://doi.org/10.1371/journal.pone.0198585
Editor: Meghan Byrne, Public Library of Science, UNITED STATES
Received: July 4, 2016; Accepted: May 22, 2018; Published: June 26, 2018
Copyright: © 2018 Théodore et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The Ministry of Education of Mexico funded this study and Bloomberg Philanthropies funded the publication of this article.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The overweight and obesity epidemic in Mexico is a public health challenge because of its extent and widespread prevalence in all age groups [1]. National studies evidenced that schools in Mexico have turned into obesogenic environments [2] where there is an extensive availability foods unhealthy food [2, 3] and scarce opportunities for physical activity [4].
Since schools are one of the institutions where children spent most of their time, the World Health Organization [5] urged countries to transform these spaces into healthy environments and where, “healthy lifestyles” have to be promoted, which include drinking plenty of water, reducing the intake of sugar-sweetened beverages, consuming fruits and vegetables, and increasing moderate to vigorous physical activity.
In 2010, the Ministries of Education and Health launched a food and physical activity policy to prevent obesity (“policy”) aimed at promoting a “new culture of healthy living, through the development of competences for a healthy life” [6], and to create a healthy environment, that include the regulation of foods sold in schools and increase children's opportunities to be physically active during the school day. Results of a evaluation showed a lack of improvement of the nutritional content of foods and beverages [7]. Therefore, an assessment of the policy implementation became necessary to understand the reason behind those results.
Historically, evaluation policy had solely focused on biological and behavioural outcomes or knowledge intermediary indicators [8–10]; thereby giving less priority to process evaluations [11, 12]. Nowadays, in the nutrition field greater attention is paid to research oriented towards informing and improving the design and implementation of interventions [13–15]. However, little is known about the elements that contribute to the success or failure of public health policies and programs in a real-life setting [15].
There are different conceptual frameworks to analyze the implementation process of a policy. The most commonly used include the intervention Mapping Protocol [16], the RE-AIM framework [17], indicators of the implementation success, or the logic model of the Logical Framework Approach and its variants [15, 18, 19]. The logic model and its variants analyze the links between inputs, products and/or processes, and the short-term and long-term results of a program (Outcomes).
The objective of this article is to assess the implementation of an obesity prevention policy in elementary schools in Mexico and learn from it to improve the implementation process.
Materials and methods
Study framework: The logic model
The logic model was used to assess the two main components of the policy: healthy food and increased physical activity. Inputs correspond to resources (personnel, structures, funds) that are allocated for the operation of the policy; products/outputs are all kinds of operations and services that must be carried out in order to make the policy works. The results/outcomes are the expected changes generated. For short terms, they can be related to awareness, knowledge, attitudes, skills, intention, and motivation, etc. For intermediate terms, they are relative to behaviors or environmental changes. For or long-term or impact, there a relative to population’s health status. The goal of this NP is to tackle obesity among schoolchildren in Mexico. The present analysis uses data from a survey requested by the Ministry of Education in order to perform an outcome evaluation of the implementation of the guidelines of foods sold in schools [20] that were introduced as part of NP. The Ministry of Education also requested information about the use of six documents distributed that included information necessary to run the Program and the functioning of the two structures created to support the implementation of the program. See S1 Table. The NP, according to the logic model and based on our interpretation of available information.
Principal inputs of the policy
The General Guidelines for Dispensing or Distribution of Foods and Beverages (‘‘Guidelines”) that establish the nutritional composition of foods and beverages authorized to be sold in elementary schools is part of this policy [21] and aimed to decrease the availability and access to energy-dense foods and beverages, to increase availability and to access to safe nutrient-dense foods (prioritizing sales of vegetables, fruits and plain drinking water) and beverages and establishing a set of instructions in terms of hygiene and safety norms.
The promotion of physical activity in schools is supported by an agreement between the Ministries of Education and Health, based on WHO recommendations that establish that children should do 30 minutes or more of physical activity per school day. The policy considers four windows of opportunity. Teachers are invited to propose games and contests during recess.
Information provided to schools.
Six documents addressed to the school actors (principals, teachers, school food vendors, committee members, parents) were developed. Two are normative and provide information about the objectives of the policy, the Committees and their respective organization and missions. The technical documents provide advices to increase healthy food preparation and physical activity in school. The aforementioned documents are key to the dissemination of normative and technical knowledge that is necessary to drive the implementation and operation of the policy. According to the Ministry of Education, all elementary school principals around the country received at the launching, a package with a set of the six documents for them and to be distributed to the others actors. The number of sets delivered to each school is ignored. There is no information about training and/or workshops, organized by the Ministry of Education previous or posterior to the launching.
Organizational and human support.
With the policy, two structures were created: a Food Committee (FC) and a Physical Activity Committee (PAC). Probably the most significant operational activity of both committees is the training of school food vendors or teachers to strengthen their knowledge and their ability to offer and/or promote healthy foods (FC) and physical activity (PAC). The FC is also in charge of the supervision of the implementation of the Guidelines. The activities of the PAC are related to the implementation and use of the physical activation guides for elementary school teachers and, the recovery of safe spaces at school and in surrounding areas for physical activity.
The policy states that FC and PAC should be made up of a majority of parents, as well as the principal and representatives of teachers and alumni.
Samples of schools and actors
The data come from a descriptive survey with national representativeness at the elementary school level, based on a three-stage stratified cluster sample. We have stratified according to school sustainment (public and private). The distribution of the sample by strata was done with a proportional allocation to the number of schools per stratum. First stage uses localities as primary sampling units selected with PPS, based on the number of schools. Second stage uses a simple random selection of 4 schools per locality and a final stage simple random selection of different school actors. In order to establish representativeness, expansion factors (sampling weights) were generated at the school level by stratum and for all different school actors considered in the survey.
We obtained in June 2012, the data from 122 public and private elementary schools.
The information presented in this article comes from questionnaires administered to 116 principals, 76 members of FC, 89 members of PAC, 132 school food vendors, 119 teachers and 348 parents without any responsibility in a FC or PAC. Not all schools had committees, but all had at least, one person covering the activities related to the FC and PAC.
In Mexico, elementary schools have two shifts. We considered both. There is no major school meal program. During recess time, vendors authorized, sell the food inside the school.
Description of the instruments used
We developed six different questionnaires for principals, Members of FC, Members of PAC, teachers, parents and school food vendors, using pre-established answer choices. Topics of the questionnaires with the stakeholders, teachers, school food vendors and parents are presented in S2. In field, we didn’t read out to the interviewees. Whenever a person provided an answer not captured by the predetermined choices, it was codified as "other". We carried out the pilot testing of the questionnaires in five schools. The pilot testing of the questionnaires was carried out in five schools.
Statistical analysis
The proportions of responses to the different variables of interest and their 95% confidence intervals were calculated. The analyses were made adjusting by the study’s design with the module of complex samples of the SPSS v18 and Stata v12 for Windows statistical packages.
Ethical considerations
The Ethics and Research Committees at the National Institute of Public Health (INSP) approved this study on June 6, 2012 (CI: 1999, No.1233). The Biosecurity Committee at the INSP considered the study exempt. All participants gave written consent to participate in the research project. The Declaration of Helsinki-Ethical Principles for Medical Research Involving Human Subjects was considered for this study.
Results
Study population
We administered questionnaires to 880 stakeholders from 122 public schools, which are representative of the 61,518 elementary schools at the national level. Table 1 describes the general characteristics of the study population.
Nearly 90% of those surveyed are from public schools. In Table 2, we summarize our main results based on the Logic Model.
Product/outputs
Availability and review of documents.
The availability of the documents among actors is low in general and varies according to the types of actor and document (Table 3).
Principals reported having a better availability to the documents, compared to other stakeholders like FC Members (21.1% Vs. 4.6% in the case of the document #1). 41.8% of school food vendors declared to have a copy of the document for preparation and hygiene of foods and beverages, but only 48% of them, had read it. Levels of review varied according to their nature and the type of actor as showed in Table 3. The principals reviewed the documents in a higher proportion, compared to members of the FC/ PAC, and the teachers. Usually, members of the FC/ PAC and teachers reviewed more the technical documents than the normative one.
The committees’ organization and operation.
49% of the elementary schools have a FC, while in schools without one, someone from the school, generally a teacher, assumes the FC functions (Table 4). These data are unavailable for the PAC.
Results/outcomes
Knowledge acquisition.
93% of principals were aware of the Guidelines and 92% believed the Guidelines were the only strategy to tackle childhood obesity in school (not shown in the table). Overall, members of the FC/ PAC, teachers and vendors showed a superficial normative and technical knowledge of the National Program (Table 4).
Indeed, principals and FC members have a partial understanding of the FC functions. They didn’t mention a large range of the functions. Usually they limited the FC functions the to the supervision of the fulfillment of Guidelines (46.1% and 81.4%, respectively). The promotion of safe/healthy environments, the consumption of plain drinking water, and the promotion of new culture of healthy living, were barely not mentioned (Table 4).
Regarding the PAC’s functions, principals have a better understanding of the committee’s promotional function. Recovering safe spaces in and around school to stimulate regular physical activity was the second PAC function that most interviewees mentioned as shown in Table 4. However, 30.2% of the functions indicated by the PAC members are not part of those established in the official documents.
Although members of the FC and principals consider that the Guidelines are a priority for this committee, their knowledge is also superficial (Table 5).
Healthy lifestyle promotion.
The majority of principals and members of FC and PAC reported that the activities that were mostly carried out at their schools by committees were related to the promotion of a correct diet and/or physical activity. However, according to committee members, there is, a low percentage of committees that have created awareness campaigns, offered trainings and assistance to the school community, or developed supporting documents.
Discussion
The creation and functioning of Committees, availability and review of documents, were not implemented as designed.
The creation of both Committees did not always happen and did not play a significant role in the training processes for vendors and teachers. They failed in the dissemination of technical and normative information among the school community members. Principals/teachers/members of committee/vendors’ knowledge acquisition about healthy eating allowed in school, and the kinds of physical activity that could be done with children, was unattained. The understanding of the normative framework of this Program was also impaired. The sharing of documents with the different types of actors was done in a patchy way and even, a large percentage of Committee members were unaware of the existence of these documents. Further, when actors received any of the documents, a vast majority reported not having read them. Based on the above, all types of actors had a superficial knowledge and understanding of the policy that, created barriers in the flow of implementation.
The lack of concrete objectives, goals and clearly stated roles that hinder the translation of the policy activities to concrete and simple actions, was on other barrier for the implementation of the policy. Most FC chose to focus on the Guidelines, leaving to a second place the promotion and training activities. In addition, the members of the FC, almost did not mention the promotion of water consumption among children as one of the FC’s main goals, despite the fact that the reduction of the consumption of sugar-sweetened beverages (SSB) is a national priority.
Despite the intention to involve a wide range of schools actors through committees as a strategy for the diffusion of information and create commitment about healthy lifestyles to be adopted, in practice, a low involvement of families, school food vendors, and children was documented in the operation and organization of the Committees.
All above failures displayed had hampered the attainment of the expected outcomes for building a culture of healthy living and the creation of a healthy environment.
Lessons learned
Based the Mexican experience and others from all over the word [22–27], we identified the following lessons:
- Theoretical framework and its importance
A framework is essential to picture the possible pathways leading to change, understand how the program will lead to results, and clearly articulate how the program activities will lead to immediate outputs and outcomes/results (clearly established process and outcome indicators), as well as track progress and make course corrections and solve problem that might arise during implementation.
- Appealing programs
The difficulty encountered by both Committees in involving parents reflects a widely reported issue in the international scientific literature [28–30] related to parents’ lack of time to participate in meetings and/or lack of resources to buy and prepare healthy foods [31] and lack of awareness of the importance of eating healthily and physical activity behaviours [5]. The policy did not adopt an attractive strategy for parents to keep them engaged and failed in convince them about its importance to children and documentation was not enough.
- Measurable objectives
The confused nature of this policy starts from the formulation of its general goal: “To promote a new culture of healthy living through the development of skills for a healthy life”. This formulation is abstract and difficult to measure. The general design of this policy is complex because of a large number of actors involved and many dimensions/goals considered.
It is noteworthy that principals/members of FC and PAC/teachers equate the Program with the Guidelines whereas its scope is much broader. Instead of establishing very broad objectives such as the promotion of healthy diets––thus leaving much room for imagination and alternative interpretations––the policy/program should formulate a few concrete, simple, and measurable objectives. Horodyska et al. [26], based on their systematic review, also identified “simplicity as a factor facilitating implementation”.
- Support stakeholders
The policy is structured in such a way that most of the operational responsibility falls upon the individual schools actors. It hinges on some big assumptions expecting that the delivery of materials will be enough to guarantee adequate implementation that translates in expected outcomes.
The policy makes no provisions to train and support schools actors in the implementation process. Supporting stakeholders and overseeing programs’ implementation is a key to success, as evidenced in a strategy for adolescent obesity prevention in the Netherlands [22]. The strategy outlines, step by step, how teachers can get involved in the program [27]. In a systematic review of scientific articles and stakeholder documents [26], authors established a list of 83 conditions for successful implementation of interventions and policies aimed at the prevention of obesity. Training for local implementers in its different forms (formal trainings, workshops, certifications, etc.) are facilitators for the adoption by the implementers of the policy.
Limitations of the study.
Several study limitations can be acknowledged. First, it was not possible to confirm the program’s theoretical grounding. We had to choose a theoretical framework (Cf. S1 Table) and organize the data accordingly to understand the implementation of the NP and be able to detect failures and successes. Second, the principal aim of the survey was to inform Ministries of Education on the evaluation of the implementation of the Guidelines [20] and the use of the materials that are part of the NP. In a context of limited resources (financial, human), we added a few questions in the original survey about the implementation of the NP and the functioning of the committees, which was for us an extraordinary opportunity to generate information with national representativeness. However, we were not able to do it with the same depth as would have been necessary to assess the implementation. Third, it would have been illuminating to complement this data with a qualitative component including school actors involved in the implementation, to explore their experience. And fourth, in a context in which stakeholders felt they were being evaluated and had to demonstrate that they implemented the Program, it was extremely difficult to collect information that could reflect, in a precise manner, the schools’ real experiences.
Conclusion
Although the Mexican government tried to tackle obesity in the school system, this study evidenced a number of difficulties that have seriously compromised the expected outcome. Major changes should be made in order to strengthen the policy.
Supporting information
S1 Table. The Program according to the logic model.
https://doi.org/10.1371/journal.pone.0198585.s001
(DOCX)
S2 Table. Summary information and tools obtained from the study.
https://doi.org/10.1371/journal.pone.0198585.s002
(DOCX)
S3 Table. Level measurement and indicators used in the evaluation.
https://doi.org/10.1371/journal.pone.0198585.s003
(DOCX)
References
- 1.
Rivera J, Campos-Nonato I, Barquera S, Gónzalez de Cossio T. Epidemiología de la obesidad en México: magnitud distribución, tendencias y factores de riesgo. In: Rivera J, Hernández M, Aguilar C, Vadillo F, C. M, editors. Obesidad en México Recomendaciones para una política de Estado. Ciudad de México: UNAM; 2012. p. 79–98.
- 2. Bonvecchio A, Théodore F, Safdie M, Duque T, Villanueva M, Torres C, et al. Contribution of formative research to design an environmental program for obesity prevention in schools in Mexico City. Salud Pública Mex. 2014;56(2):S139–47. pmid:25629246.
- 3. Safdie M, Lévesque L, González-Casanova I, Salvo D, Islas A, Hernández-Cordero S, et al. Promoting healthful diet and physical activity in the Mexican school system for the prevention of obesity in children. Salud Pública de México. 2013;55(3):S357–S73.
- 4. Jennings-Aburto N, Nava F, Bonvecchio A, Safdie M, Gonzalez-Casanova I, Gust T, et al. Physical activity during the school day in public primary schools in Mexico City. Salud Pública Mex. 2009;51(2):141–7. pmid:19377741
- 5.
World Health Organization. School policy framework: implementation of the WHO global strategy on diet, physical activity and health. WHO, 2008.
- 6.
SEP, SSA. Action program in the school context "Programa de Acción en el Contexto Escolar". Secretaria de Educación Pública y Secretaria de Salud, 2010.
- 7. Jiménez-Aguilar A, Morales-Ruán C, López-Olmedo N, Théodore F, Moreno-Saracho J, Tolentino-Mayo L, et al. The fight against overweight and obesity in school children: Public policy in Mexico. Journal of Public Health Policy. 2017;38(4):407–28. pmid:28860550
- 8. Waters E, Silva-Sanigorski A, Burford B, Brown T, Campbell K, Gao Y, et al. Interventions for preventing obesity in children. The Cochrane Collaboration, 2011 Contract No.: 12.
- 9. Flynn M, McNeil D, Maloff B, Mutasingwa D, Wu M, Ford C, et al. Reducing obesity and related chronic disease risk in children and youth: a synthesis of evidence with 'best practice' recommendations. Obesity reviews: an official journal of the International Association for the Study of Obesity. 2006;7 Suppl 1:7–66. pmid:16371076.
- 10. Harachi T, Abbott R, Catalano R, Haggerty K, Fleming C. Opening the black box: using process evaluation measures to assess implementation and theory building. American journal of community psychology. 1999;27(5):711–31. pmid:10676545
- 11.
Consejo Nacional de Evaluación. Evaluación de la política social: Consejo Nacional de Evaluación; [updated 25 Marzo 2015]. Available from: http://www.coneval.gob.mx/evaluacion/Paginas/Evaluacion.aspx
- 12. Harnack L, Snyder P, Story M, Holliday R, Lytle L, Neumark-Sztainer D. Availability of a la carte food items in junior and senior high schools: a needs assessment. Journal of the American Dietetic Association. 2000;100(6):701–3. pmid:10863576.
- 13. Garrett J. Improving results for nutrition: a commen- tary on an agenda and the need for implementation research. The Journal of nutrition. 2008;138(3):646–50. pmid:18287382
- 14. Menon P, Covic N, Harrigan P, Horton S, Kazi N, Lamstein S, et al. Strengthening implementation and utilization of nutrition interventions through research: a framework and research agenda. Annals of the New York Academy of Sciences. 2014;1332:39–59. pmid:24934307
- 15. Habicht J, Pelto G. From biological to program efficacy: promoting dialogue among the research, policy, and program communities. Advances in nutrition. 2014;5(1):27–34. pmid:24425719; PubMed Central PMCID: PMC3884096.
- 16. Van Nassau F, Singh A, Hoekstra T, Van Mechelen W, Brug J, Chinapaw M. Implemented or not implemented? Process evaluation of the school-based obesity prevention program DOiT and associations with program effectiveness. Health education research. 2016;31(2):220–33. pmid:27004906
- 17. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research. 2011;38(2):65–76. pmid:20957426
- 18. Saunders R, Evans M, Joshi P. Developing a process-evaluation plan for assessing health promotion program implementation: a how-to guide. Health promotion practice. 2005;6(2):134–47. pmid:15855283.
- 19. Avula R, Menon P, Saha K, Bhuiyan M, Chowdhury A, Siraj S, et al. A Program Impact Pathway Analysis Identifies Critical Steps in the Implementation and Utilization of a Behavior Change Communication Intervention Promoting Infant and Child Feeding Practices in Bangladesh. The Journal of nutrition. 2013.
- 20.
Agreement by which the General Guidelines for the sale and distribution of food and beverages prepared and processed established in schools in the national education system ""Acuerdo mediante el cual se establecen los Lineamientos generales para el expendio y distribución de alimentos y bebidas preparados y procesados en las escuelas del Sistema Educativo Nacional", (2014).
- 21. Charvel S, Cobo F, Hernández-Ávila M. A process to establish nutritional guidelines to address obesity: Lessons from Mexico. Journal of Public Health Policy. 2015;36:426–39 pmid:26356071
- 22. Van Nassau F, Singh A, van Mechelen W, Paulussen T, Brug J, Chinapaw M. Exploring facilitating factors and barriers to the nationwide dissemination of a Dutch school-based obesity prevention program "DOiT": a study protocol. BMC Public Health. 2013;13:1201. pmid:24355056; PubMed Central PMCID: PMC3878240.
- 23. Blaine R, Franckle R, Ganter C, Falbe J, Giles C, Criss L, et al. Using School Staff Members to Implement a Childhood Obesity Prevention Intervention in Low-Income School Districts: the Massachusetts Childhood Obesity Research Demonstration Preventing chronic disease. 2017;14(160381). https://doi.org/10.5888/pcd14.160381.
- 24. Osganian S, Parcel G, Stone E. Institutionalization of a school health promotion program: background and rationale of the CATCH-ON study. Health Education & Behavior. 2003;30(4):410–7.
- 25. Van Lippevelde W, Verloigne M, De Bourdeaudhuij I, Brug J, Bjelland M, Lien N, et al. Does parental involvement make a difference in school-based nutrition and physical activity interventions? A systematic review of randomized controlled trials. International journal of public health. 2012;54(4):673–8.
- 26. Horodyska K, Luszczynska A, Hayes C, O’Shea M, Langøien L, Roos G, et al. Implementation conditions for diet and physical activity interventions and policies: an umbrella review. BMC Public Health. 2015;15(1250). pmid:26678996
- 27. Singh A, Chinapaw M, Kremers S, TLS. V, Brug J, van Mechelen W. Design of the Dutch Obesity Intervention in Teenagers (NRG-DOiT): systematic development, implementation and evaluation of a school-based intervention aimed at the prevention of excessive weight gain in adolescents. BMC Public Health. 2006;6(304).
- 28.
Morales F. Participación de padres en la escuela: Componente para la formación de profesores Chile: CIDE, 1998.
- 29. Epstein J, Becker J. Teachers' Reported Practices of Parent Involvement: Problems and Possibilities. The Elementary School Journal. 1982;83(2).
- 30. Cotton K, Wikelund K. Parent involvement in education. School Improvement Research. 1989;6.
- 31. Olivares S, Lera L, Mardones MA, Araneda J, Olivares MA, Colque ME. [Motivation and barriers in the consumption of five daily servings of fruit and vegetables by mothers of school age children and primary school teachers]. Archivos latinoamericanos de nutricion. 2009;59(2):166–73. pmid:19719013.