Our Baseline/Endline Survey Methodology
1) To determine community knowledge about the project and perception of project benefits.
2) To determine community knowledge about monitoring and specifically, community-based monitoring.
3) To determine community knowledge about implementers and donors, including who they perceive to be responsible for monitoring projects.
4) To determine community perceptions of corruption in projects.
5) To determine community access to information (e.g. scope of work, BoQ, drawings).
6) To collect demographic data on survey respondents (data should be disaggregated by sex and age).
In the baseline and end-line surveys of CBM, the representative sampling methodology of Random Sampling will be used. All individuals in the sampling frame will have equal probability of being selected in the sample. The community where the infrastructure project is implemented will serve as a Primary Sampling Unit (PSU). Then, the PSU will be divided by the FP/Program Officer into 20 sub units. From these 20 sub units 10 of them will be selected randomly to conduct the interviews. In these 10 sub units 10 interviews will be conducted into each of these sub units to make a total of 60 interviews in this community. The same methodology of respondent selection will be applied in the Endline survey as well.
SURVEY TIME FRAME
Baseline: Conducted by Local Monitors, supervised by Focal Points in the first month of monitoring.
Endline: Conducted by Local Monitors, supervised by Focal Points in the month after project completion.
Local Monitors should be introduced to the baseline survey during the LM Training workshop. This introduction MUST be followed up by an in-depth training by Focal Points/Program Officer on how to administer a baseline survey. We must use a simple, baseline survey methodology guide.
DATA GATHERING TECHNIQUES
Though surveys can be conducted using many techniques, such as Paper Surveys, Phone Surveys, Mail Surveys and PDA-Based Surveys. Each of these surveys have their own merits, in terms of cost, time required, security constraints and errors in data entry. Though the paper-based survey requires a big budget, a lot of time and has a lot of logistical constraints, it is the most popular survey form in Afghanistan.
Here are some helpful suggestions for survey design. For more in-depth information on survey design methodology, you can contact the Integrity Watch office and we can share our experiences.
Order of Questions
First: Basic questions (e.g. age, address, level of education) should be in the beginning.
Second: Friendly questions that are non-threatning should be next to relax the respondent.
Third: The most tricky and complex questions should be in the middle.
Fourth: The questions that may threaten the relationship between the surveyor and the respondent should be last (e.g. questions about personal, sensitive issues, opinions, corruption)
Before starting with the real survey, it is important to test the questionnaire to determine if the survey has any mistakes, is unclear or has any other problems. Once the questionnaire is drafted, it is useful to organize a focus group with 6 to10 people who are familiar with the studied subject or other colleagues who can constructively criticize the survey. The test will help identify the following things:
1) Problems regarding the formulation of questions.
2) The lack of suitable responses.
3) The repetition of certain questions.
4) The time necessary to complete the survey.
Once the focus group has identified problems and these have been fixed, surveyors can test the questionnaire on a small scale, for example with 30 to 50 respondents. This type of test is usually sufficient to detect the major mistakes. The surveyors will then report the problems they faced with the real respondents.
The focus group and test surveyors will help keep refining the questionnaire until an agreement is found and you can finalize the survey. This does not mean that the questionnaire will be perfect, but the major issues should be detected. This series of tests is crucial before the survey is implemented on a full-scale.
It is also wise to analyze the results of the pre-test, because sometimes the questionnaire can be well structured, easy to implement in the field, but it may not be appropriate for answering the research questions. Therefore, when we conduct the pre-test, the data should also be analyzed in order to ensure that the survey can answer our research questions.
DATA INPUT & ANALYSIS
After the survey is completed, you will need to hire data-entry officers and a data analyst on a short-term or permanent basis. Data input requires data cleaning (e.g. discarding incomplete survey forms), data coding and most importantly, setting up a database to collect all the data. The data analyst should conduct quantitative data analysis using statistical analysis (SPSS) software. She/He will then analyze the data to produce frequency distributions, cross-tabulation and multiple response ranking. This toolkit does not go in depth on the analysis process because data analysis requires academic and professional training and should not be done without proper training. This is because it is easy to manipulate and misinterpret data. Subjective & biased data analysis is thus, often used as a political tool. This should always be avoided.