Human Resources: Building the Survey Team Suleiman Asman
Lecture Overview • Planning • In- House Survey: Hiring and Contracting • In- House Survey: Training • In- House Survey: Managing survey team • In House vs. Outsourcing • Outsourcing process
J-PAL | BUILDING THE SURVEY TEAM
2
Lecture Overview • Planning • In- House Survey: Hiring and Contracting • In- House Survey: Training • In- House Survey: Managing survey team • In House vs. Outsourcing • Outsourcing process
J-PAL | BUILDING THE SURVEY TEAM
3
Planning Local context
Scope of the study
Budget and Timelines
Survey Plan
Key Decisions
Paper vs. Electronic J-PAL | BUILDING THE SURVEY TEAM
Permissions
In-house vs. Outsourcing 4
Planning: Local Context •
Terrain
•
Electricity
•
Topography
•
Holidays and major events
•
Availability of respondents
•
Transport options
•
Pool of potential surveyors
•
Survey printing options
•
Internet
•
Storage
•
Office space
•
Training venues
•
Gifts
•
Survey equipment
•
Stationary
•
Permissions and authorizations
•
Ability to recruit subject pool for study
J-PAL | BUILDING THE SURVEY TEAM
5
However……… There are always unexpected events You should expect them!
• Buffer – Time allocation – Staff allocation
J-PAL | BUILDING THE SURVEY TEAM
6
Lecture Overview • Planning • In- House Survey: Hiring and Contracting • In- House Survey: Training • In- House Survey: Managing survey team • In House vs. Outsourcing • Outsourcing process
J-PAL | BUILDING THE SURVEY TEAM
7
Survey Team Structure
Sample size
Budget
J-PAL | BUILDING THE SURVEY TEAM
Duration
8
Survey Team Structure Position
Responsibilities
Research Associate
Field Staff Coordinator, logistics, finances, hiring
Field Manager
Monitor field operations and finances of many teams, field logistics, surveyors recruitments, organize trainings
Associate Field Manager
Daily support for supervisors , productivity, and monitoring
Supervisors/Senior field officers
Team management and field quality checks
Surveyors/enumerator
Data collection and first round of edits
Editors/Scrutinizers
Scrutiny surveys (paper only)
Auditors/Back Checkers
Back checks
Trackers
Help track hard to find respondents
J-PAL | BUILDING THE SURVEY TEAM
9
Survey Team Recruitment • Clear sense of the project and its specific requirements • Clarity on numbers and payments • Survey team structure and composition
J-PAL | BUILDING THE SURVEY TEAM
10
How to Recruit? •
Hire locally if possible (Plan ahead)
•
Advertising – Where are we advertising? • Modes and risks
•
Numbers – Receive 10 times as many as you need – Test 5 times as many as you need – Start training 30% more than you need – Hire 20% more than you need (attrition) – Be ready to fire
J-PAL | BUILDING THE SURVEY TEAM
11
Selection Process • Preliminary selection – First level screening – CVs and Resumes • Interviews and Tests – Conduct rigorous interviews – Testing computer/language skills – Letters of recommendation
J-PAL | BUILDING THE SURVEY TEAM
12
Examples of Tests • Verbal tests in all relevant languages – Reading out loud – Reading and summarizing/explaining out loud • Role play • Situational judgement tests • Digital literacy • Tests for higher level staff – Design a simple budget and/or field plan (PAs, FMs) – Excel tests (PAs, FMs) – Math/logic tests
J-PAL | BUILDING THE SURVEY TEAM J-PAL | BUILDING THE SURVEY TEAM
13
Selection Criteria Qualities to look for in a candidate: – Technical capacity – Interest – Communication Skills – Attentiveness – Attitude – Confidence – Trustworthiness – Body language • Candidate expectations • Performance during training J-PAL | BUILDING THE SURVEY TEAM
14
At the time of Hiring •
Clearly convey the roles and responsibilities of all the field staff – Project specific information
•
Confidentiality agreements
•
Discuss all administrative formalities – Contracting – Payments – Travel reimbursements – Attendance – Advances – Performance standards – ID’s, insurance claim
•
Clarify any queries on their contracts, salary structure etc.
•
Do NOT START THE SURVEY before the contracts are issued
J-PAL | BUILDING THE SURVEY TEAM
15
Lecture Overview • Planning • In- House Survey: Hiring and Contracting • In- House Survey: Training • In- House Survey: Managing survey team • In House vs. Outsourcing • Outsourcing process
J-PAL | BUILDING THE SURVEY TEAM
16
Training field staff: Activities • Survey manual • Logistics • Training the leaders • Human subjects • Role clarity • Review of the instrument • Interviewing style • Technical training • Safety and security • Field training
J-PAL | BUILDING THE SURVEY TEAM
17
Survey Manual • Follows the same structure as the survey instrument • Detailed instructions for administering the survey • Serves as a guide for training • Should be updated during training as complications are revealed
J-PAL | BUILDING THE SURVEY TEAM
18
Contents of Survey Manual Survey Manual Training content is available in the next unit.
TABLE OF CONTENTS BACKGROUND INFORMATION DEFINITION OF TERMS EXIT SURVEY AT HEALTH FACILITY INTRODUCTION: SCREENING QUESTIONS FULL EXIT INTERVIEW QUESTIONS MODULE 2: HOUSEHOLD LOCATION DATA COLLECTION HOUSEHOLD SURVEY THE CONSENT MODULE 1: ROSTER/DEMOGRAPHICS SECTION 1: HOUSEHOLD DEMOGRAPHICS SECTION 2: HOUSEHOLD DEMOGRAPHICS (MARITAL STATUS AND EDUCATION) SECTION 3: EMPLOYMENT SECTION 4 UNDER FIVE: SECTION 5: HOUSEHOLD RELIGION AND LANGUAGE MODULE 2: HOUSEHOLD CHARACTERISTICS SECTION 1: QUESTIONS FOR THE HOUSEHOLD TYPES OF TOILETS SECTION 2: MOSQUITO NET USE MODULE 3: PREGNANCY HISTORY, CHILD HEALTH, AND UTILIZATION MODULE 4: HOUSEHOLD ASSETS SECTION 4: . HOUSEHOLD FOOD CONSUMPTION MODULE 5: OBSERVATION OF THE DWELLING STRUCTURE AND THE SURROUNDING BASICS ON THE USE OF GPS UNITS SETTING THE SCENE FOR AN INTERVIEW RESEARCH ETHICS PROTECTING PARTICIPANTS RIGHTS IN RESEARCH MAIN PRINCIPLES OF RESEARCH ETHICS RESPONDENTS GIFT J-PAL | BUILDING THE SURVEY TEAM
19
Training Logistics • Training location • Food • Compensation for training • Training material Standard time frame including field dry-runs is 2 weeks to 2 months
J-PAL | BUILDING THE SURVEY TEAM
20
Training the Leaders • Trained with surveyor or trained beforehand •
Management - Come with experience - Trained on management
J-PAL | BUILDING THE SURVEY TEAM
21
Training on Ethics • Adequate training on human subjects • Thorough review of the consenting process
J-PAL | BUILDING THE SURVEY TEAM
22
Role Clarity • Role of a surveyor – Locate respondent, as per supervisors advice – Establish trust, motivate and encourage respondent – Get consent – Conduct survey, – Clarify any confusion or concerns – Ensure quality data collection – Report to supervisor – Maintain code of conduct and other reporting protocols • Similarly, clarify other roles during the training J-PAL | BUILDING THE SURVEY TEAM
23
Review of the Instrument • Crux of the training process • Section-by-section, question-by-question instruction • Use the survey manual
J-PAL | BUILDING THE SURVEY TEAM
24
Interviewing style: Standardized interviews •
Read questions exactly as worded – Neutral or non-directive probing
Pros
Cons
• Faster interviews
• May lead to misunderstanding and inaccurate answers • Should, in principle, eliminate surveyor effects or bias • Can be sure all respondents receive same materials Source: Conrad, Frederick and Frauke Kreuter (2014), ‘Questionnaire Design for Social Surveys’, Available at Coursera
J-PAL | BUILDING THE SURVEY TEAM
25
Interviewing style: Conversational Interviews •
Allow Interviewer to say whatever it takes to be sure that the respondent understands question as intended Pros
• Question is clearly conveyed to respondent
Cons • Time consuming • Potential for surveyor effects
Source: Conrad, Frederick and Frauke Kreuter (2014), ‘Questionnaire Design for Social Surveys’, Available at Coursera
J-PAL | BUILDING THE SURVEY TEAM
26
Interviewing style: Making a Choice • Depends on the purpose and goals of the research – What information is needed? • Tradeoff between longer and maybe more costly interviews against higher certainty that respondents understand questions as intended • Can also vary per module/question for the same survey • Either way, surveyors need to be trained appropriately on how to execute the questionnaire Source: Conrad, Frederick and Frauke Kreuter (2014), ‘Questionnaire Design for Social Surveys’, Available at Coursera
J-PAL | BUILDING THE SURVEY TEAM
27
Technical Training • Digital literacy • Specific measurement tools – Health related, GPS devices etc.
• Invite specialists for technical training • Consider training on paper before using digital data collection devices
J-PAL | BUILDING THE SURVEY TEAM
28
Safety and Security • Road safety • Safety while in the field (dress code, working hours etc.) • Safely and security of equipment (inc. gifts and compensation) • Sexual harassment policies
J-PAL | BUILDING THE SURVEY TEAM
29
Role play in Classroom Settings • Role play – Teaming up as surveyor and respondent – Go through entire survey – Spend about 2-3 days on practicing in teams
• Go to field after team is clear about survey – E.g. how to deal with standardized vs conversational styles
J-PAL | BUILDING THE SURVEY TEAM
30
Field interviews: Training and Piloting • Field practice in conditions/sample similar to the actual survey – Use separate sample
• Evaluate and make final selection of surveyors – Also team leaders
• False launch – Challenge: finding separate sample – Use pre-existing lists
• Daily debrief – From surveyors AND supervisors
J-PAL | BUILDING THE SURVEY TEAM
31
Lecture Overview • Planning • In- House Survey: Hiring and Contracting • In- House Survey: Training • In- House Survey: Managing survey team • In House vs. Outsourcing • Outsourcing process
J-PAL | BUILDING THE SURVEY TEAM
32
Setting Performance Expectations •
Have clear rules to begin with – Quality is more important than efficiency
•
Communicate clearly and regularly
•
Stress on punctuality
•
Surveyors’ behavior with respondents
•
Should be strict and consistent
•
Examples of zero-tolerance offences – Confidentiality breaches – Misbehavior with respondents/co-surveyors – Falsification of data
J-PAL | BUILDING THE SURVEY TEAM
33
Setting Productivity Goals • Surveyor productivity – Reasonable targets • Some productivity factors (quality) 1. Daily travel to the field 2. Locating respondents 3. Duration of the survey
J-PAL | BUILDING THE SURVEY TEAM
34
Motivating Surveyors • Financial incentives – Timely payments – Preferably don’t pay by survey – End of field work bonuses • Career and growth – Promotion and retention – Professional development • Non-financial incentives – Be a good manager, develop interpersonal skills – Be part of the team: Go with them, stay with them – Fun outings / Team building • Be apologetic about any last minute field changes and explain them the reasons for doing so. J-PAL | BUILDING THE SURVEY TEAM
35
Lecture Overview • Planning • In- House Survey: Hiring and Contracting • In- House Survey: Training • In- House Survey: Managing survey team • In House vs. Outsourcing • Outsourcing process
J-PAL | BUILDING THE SURVEY TEAM
36
Hiring a company?
Pros
Cons
• Logistics
• Quality standards
• Human resources
• Intensive monitoring
• Local knowledge
• Complex negotiations
• Predictable costs
• Can be more expensive
J-PAL | BUILDING THE SURVEY TEAM
37
Making a Choice • Timeline and number of surveyors needed – Who can recruit surveyors more quickly? – How long will it take to review proposals, negotiate contracts?
• Budget • Location and experience of firms in study area • Personal contacts in the local area • Amount of time for planning J-PAL | BUILDING THE SURVEY TEAM
38
Lecture Overview • Planning • In- House Survey: Hiring and Contracting • In- House Survey: Training • In- House Survey: Managing survey team • In House vs. Outsourcing • Outsourcing process
J-PAL | BUILDING THE SURVEY TEAM
39
Steps in outsourcing • Plan as if you were going to do a survey on your own • Identifying and contacting companies - Local presence - Share project details -
Request proposals and quotes from interested companies
J-PAL | BUILDING THE SURVEY TEAM
40
Components of Project Details • • • • • • • • • •
Timeline, including a deadline for proposals Approximate number of surveyors needed The size of your sample The survey instruments and length Respondents revisits/replacement rules Gender ratio and minimum ratio of experienced surveyors you require Reporting requirements and frequency Quality controls Requirement of regular re-training/feedback sessions Your independent quality check process
J-PAL | BUILDING THE SURVEY TEAM
41
Components of quotes from companies • • • • • • • • • • •
Composition of the research team, with titles Surveyor to supervisor ratio Educational level of the team members Level of field experience of each member Permanent to temporary worker ratio Timeline Quality controls Deliverables Previous experience with surveys in the study areas, References of prior clients Cost estimates, and breakdowns
J-PAL || BUILDING BUILDING THE SURVEY TEAM J-PAL THE SURVEY TEAM
42
Comparison •
Compare the quotes – Seek clarification where required
•
Things to look at carefully in a budget: – Surveyors salaries – Field operations budget – Non operating costs, such as profit margins, overheads, researchers’ time, etc.
•
Be careful: a company can also UNDER-bid, in particular when it’s a big survey
J-PAL | BUILDING THE SURVEY TEAM
43
Bidding • Lowest quote is not necessarily the best quote – Cost is only one of many factors • Technical expertise – Past experiences • Quality control measures • Professionalism
J-PAL || BUILDING BUILDING THE SURVEY TEAM J-PAL THE SURVEY TEAM
44
Contract • Include rules for all relevant scenarios – Independent monitoring – Strategies to deal with: • • • •
Respondents not available Survey delay Poor performing (or dishonest) surveyors Changes in scope of the study
• Penalties
J-PAL || BUILDING BUILDING THE SURVEY TEAM J-PAL THE SURVEY TEAM
45
Working with companies • Pilot • Training • Quality checks
J-PAL | BUILDING THE SURVEY TEAM
46
END