PISA, FAQ

What does the term "PISA" mean? PISA is an acronym taken from the "Programme for International Student Assessment". What is the history of PISA? Responding to member countries’ demands for regular and reliable data on the knowledge and skills of their students and the performance of their education systems, the OECD began work on PISA in the mid-1990s. PISA was officially launched in 1997, with the first survey taking place in 2000, the second in 2003, the third in 2006 and the fourth in 2009. Further surveys are planned in 2012, 2015 and beyond… What makes PISA unique? PISA benefits from its worldwide scope and its regularity. More than 70 countries and economies have taken part in PISA so far and the surveys, which are given every three years, allow them to track their progress in meeting key learning goals. PISA is the only international education survey to measure the knowledge and skills of 15-year-olds, an age at which students in most countries are nearing the end of their compulsory time in school. PISA is also unique in the way it looks at: - Public policy issues Governments, principals, teachers and parents all want answers to questions such as "Are our schools adequately preparing young people for the challenges of adult life?", "Are some kinds of teaching and schools more effective than others?" and "Can schools contribute to improving the futures of students from immigrant or disadvantaged backgrounds?" - Literacy Rather than examine mastery of specific school curricula, PISA looks at students’ ability to apply knowledge and skills in key subject areas and to analyse, reason and communicate effectively as they examine, interpret and solve problems. - Lifelong learning Students cannot learn everything they need to know in school. In order to be effective lifelong learners, young people need not only knowledge and skills, but also an awareness of why and how they learn. PISA both measures student performance in reading, mathematics and science literacy and also asks students about their motivations, beliefs about themselves and learning strategies. Which countries and economies participate in PISA? All OECD member countries participated in the first three PISA surveys, along with certain partner countries and economies. In total, 43 countries and economies took part in PISA 2000, 41 in PISA 2003, 58 in PISA 2006 and 74 in PISA 2009. For PISA 2009, 65 countries/economies implemented the assessment in 2009 (with results published on 7 December 2010). A further 9 countries/economies implemented the same assessment in 2010. Results for these 9 countries/economies will be published in December 2011. Click here to obtain the full list of PISA participants to date and information on each country’s contacts, website, national report and more. How are countries/economies chosen to participate in PISA? Countries and economies  interested in participating in PISA contact the OECD Secretariat. The PISA Governing Board then approves membership according to certain criteria. Participating countries and economies must have the technical expertise necessary to administer an international assessment and must be able to meet the full costs of participation. To take part in a cycle of PISA, participants must join two years before the survey takes place. For example, all countries participating in PISA 2012 will have joined before March 2010. Who are the people behind PISA? Education ministries in the participating countries/economies PISA would not be possible without the support and guidance of the education ministries in the participating countries. Click here for links to education ministries in OECD countries. The OECD Secretariat The OECD Secretariat is responsible for the day-to-day management of PISA. This means that the PISA team monitors the survey’s implementation, manages administrative matters for the PISA Governing Board, builds consensus among countries and serves as a go-between for the PISA Governing Board and the PISA Consortium. Click here to see the OECD Secretariat contact list for PISA. The PISA Governing Board Each OECD country participating in PISA has a representative on the PISA Governing Board, appointed by the country’s education ministry. Each partner country participating in PISA has an observer on the board, appointed by the country’s education ministry. The chair of the PISA Governing Board is chosen by the Board itself. Guided by the OECD’s education objectives, the Board determines the policy priorities for PISA and makes sure that these are respected during the implementation of each PISA survey. Click here for the names and contact details of the PISA Governing Board. The international contractors (the "PISA Consortium") For each PISA survey, international contractors (usually made up of testing and assessment agencies) have been responsible for the design and implementation of the surveys. The contractors are chosen by the PISA Governing Board through an international call for tender. This contractors are referred to as the "PISA Consortium". The PISA National Project Managers Working with the OECD Secretariat, the PISA Governing Board and the international contractor, the PISA National Project Managers oversee the implementation of PISA in each participating country/economy. The PISA National Project Managers are appointed by their governments. The PISA Subject Matter Expert Groups PISA has Subject Matter Expert Groups for its three key areas of testing – reading, mathematics and science literacy – as well as for other subjects when appropriate (problem solving in PISA 2003, for example). These groups are made up of world experts in each area. They design the theoretical framework for each PISA survey. The PISA Questionnaire Expert Group The Questionnaire Expert Group provides leadership and guidance in the construction of the PISA context questionnaires. The members of the Questionnaire Expert Group are selected by the PISA Governing Board. Who pays for PISA? PISA is financed exclusively through direct contributions from government authorities of the participating countries and economies (typically Education ministries). Does PISA tell countries and economies how to run their schools? No. The data collected by PISA shows the successes of some countries’ schools and the challenges being faced in other countries. It allows countries to compare best practices and to further develop their own improvements, ones appropriate for their school systems.

Who takes the PISA tests? Schools in each country/economy are randomly selected by the international contractors for participation in PISA. At these schools, the test is given to students who are between age 15 years 3 months and age 16 years 2 months at the time of the test, rather than to students in a specific year of school. This average age of 15 was chosen because at this age young people in most OECD countries are nearing the end of compulsory education. The selection of schools and students is kept as inclusive as possible, so that the sample of students comes from a broad range of backgrounds and abilities. What does PISA test? Every PISA survey tests reading, mathematical and scientific literacy in terms of general competencies, that is, how well students can apply the knowledge and skills they have learned at school to real-life challenges. PISA does not test how well a student has mastered a school’s specific curriculum. How does PISA test this? To date, PISA has used pencil-and-paper tests. Some countries/economies also took part in an option to implement assessments using computers in 2006 and 2009. The tests are made up of both multiple-choice questions and questions requiring students to construct their own responses. The material is organised around texts and sometimes includes pictures, graphs or tables setting out real-life situations. Each PISA survey includes about seven hours of test material. From this, each student takes a two-hour test, with the actual combination of test materials different for every student. Who creates the test questions? All PISA countries/economies are invited to submit questions to the international contractors; in addition, the international contractors also write some questions. The questions are reviewed by the international contractors and by participating countries/economies and are carefully checked for cultural bias. Only those questions that are unanimously approved are used in PISA. Further, before the real test there is a trial run in all participating countries/economies. If any test questions prove to have been too easy or too hard in certain countries/economies, they are dropped from the real test in all countries/economies. What are the PISA context questionnaires? What are they used for? Students answer a background questionnaire, providing information about themselves, their attitudes to learning and their homes. It takes 20-30 minutes to complete. In addition, school principals are given a 20-minute questionnaire about their schools. Countries can also choose to administer several optional PISA questionnaires: the computer familiarity questionnaire, the educational career questionnaire and the parent background questionnaire. In addition, many countries choose to gather further information through national questionnaires. The information collected from these questionnaires helps countries to explore connections between how students perform in PISA and factors such as migration, gender and students’ socio-economic background, as well as students’ attitudes about school and their approaches to learning. Are the PISA tests available to the general public? A selection of PISA test materials is available in the publication PISA Take the Test: Sample Questions from the OECD's PISA Assessments (2009) Why aren’t all the PISA test materials available? In order to allow countries/economies to follow their performance over time, many questions are used in more than one PISA survey. These questions cannot be made public as long as they are in use.

How are the tests corrected? Each country and economy has its own group of test correctors, overseen by their National Project Manager. They mark the PISA tests using a guide developed by the international contractors and the PISA Subject Experts (with input from all participating countries and economies). The corrections are cross-checked by other experts. The final results are then sent to the international contractors, who in turn transmit the final data to the OECD Secretariat. What do the test scores mean? PISA scores can be located along specific scales developed for each subject area, designed to show the general competencies tested by PISA. These scales are divided into levels that represent groups of PISA test questions, beginning at Level 1 with questions that require only the most basic skills to complete and increasing in difficulty with each level. Once a student’s test has been corrected, his or her score in reading, mathematics and science (plus problem solving in PISA 2003) can be located on the appropriate scale. For example, a student who is likely to lack the skills need to correctly complete easiest questions on a PISA test would be classified as below Level 1, while a student who is likely to have many of these skills need to correctly complete the test questions would be at a higher level. In each test subject, the score for each participating country and economy is the average of all student scores in that country. The average score among OECD countries is 500 points and the standard deviation is 100 points. About two-thirds of students across OECD countries score between 400 and 600 points. Where can I find more information on the PISA scales? These are described in detail in the PISA assessment framework publications and the PISA technical reports. For free PDFs of these books, click here How are countries and economies ranked in PISA? PISA ranks countries and economies according to their performance in reading, mathematics and science, as well as problem solving in PISA 2003. PISA does not give a collective score for all subjects combined; rather it gives a score for each subject area and countries/economies are ranked by their mean score in each area. However, it is not possible to assign a single exact rank in each subject to each country/economy. This is because PISA tests only a sample of students from each country/economy and this result is then adjusted to reflect the whole population of 15-year-old students in that country/economy. The scores thus reflect a small measure of statistical uncertainty and it is therefore only possible to report the range of positions (upper rank and lower rank) within which a country can be placed. For example, in PISA 2003 Finland and Korea were widely reported as ranking 1st and 2nd in PISA, when in fact we can only say that, among OECD countries, Finland’s rank was between 1st and 3rd and Korea’s was between 1st and 4th . Can results from PISA be compared across the surveys or does each survey stand alone? Yes, student performance can be compared across the surveys, as can some of the background questionnaire items. Are the data from the PISA surveys publicly available? Yes. Click here  to access the databases for the assessments carried out in 2000, 2003, 2006 and 2009. Are the results of every country and economy's performance in PISA published? Generally, yes – the PISA reports published after each survey include data from all participating countries/economies as long as the data and the amount of students tested (the sample size) meet certain standards. If the standards are not attained, depending on the reason, the data are either not published at all or published but set apart from data of other countries with an explanatory note .
For the PISA 2000 cycle, the response rate from schools in the Netherlands was too low for its data to be comparable. The same was true for the United Kingdom for the PISA 2003 cycle. Data for these countries were published but set apart from other countries with an explanatory note.
For the PISA 2006 cycle, in the United States an error in printing the test booklets, in which the pagination was changed and instructions for some reading items directed students to the wrong page, may have affected student performance. The potential impact of the printing error on student performance was estimated by examining the relative performance of students in the United States on the item set that was common between PISA 2006 and PISA 2003, after controlling for performance on the items that were not likely to be affected by the printing error.
The predicted effect of the printing error and the wrong directions on student mean performance on the reading test was up to 6 score points, and thus exceeds one standard error of sampling. Reading performance data for the United States are therefore excluded from this publication and the PISA database.
The predicted effect of the printing error on student mean performance on the mathematics and science tests was one score point. Mathematics and science performance data for the United States, were therefore retained.

How can I learn more about PISA? Click here to access all PISA products Click here  to go to the PISA country profiles, a free, interactive online service that allows you to take a close look at countries’ performances in PISA 2003 and PISA 2006 . Click here to visit the OECD’s Directorate for Education. I have a question about PISA and I haven’t found the information I need in this FAQ We welcome questions and suggestions for areas that need further explanation. Please send us your comments and queries at pisa.contact@oecd.org. We cannot promise to answer all individual messages, but this site will be regularly updated to reflect your input.