Data Collection Procedures
Upon completion of this chapter, you will be able to:
- Study the two data sources, primary and secondary, for research
- Learn to search for secondary data through computer systems
- Know about computer search through the WWW
- Understand that mixing primary and secondary data is tricky and that care must be exercised
- Learn the principles of scientific observation
- Learn to design questionnaires
- Understand how to conduct interviews
- Familiarise yourself with method and kinds of questions that are asked in research
- Understand the errors that occur in data collection
- Learn to validate the various methods of data collection
Data required for management research can be classified into primary data and secondary data. Primary data is the data specially collected in a research by the researcher and his agents. These are the products of experiments, surveys, interviews, or observations conducted in the research. Secondary data is data collected by other researchers for their own use and which is of use to another research project. Primary data is generated and collected through specific tools of data collection, like questionnaires, by the researcher. Secondary data is searched for and obtained from many different sources. The main effort involved in acquiring secondary data is searching for and locating it, which is increasingly becoming a specialised and skilled task in the present context of information explosion and the advent of complex computer search systems.
Acquiring secondary data is less expensive and less time consuming than collecting primary data. This is the biggest advantage with secondary data. But often problems arise while using secondary data.
acquiring secondary data
The main effort of acquiring secondary data is searching for and locating it. Acquiring secondary data is less expensive and less time consuming but has problems: it may not meet exactly the requirements of the research; details may not be available and validating the data may be difficult.
- Secondary data may not meet the needs of the problem at hand for various reasons. Units of measurement may be different; aggregated data often does not provide the details required by the researcher; classifications of data may not match the requirements of the problem at hand; and the period for which the data are available may not suit the overall research needs.
- All the data required for research analysis may not be available. Matching part data obtained by primary data collection methods and part by secondary sources may become extremely difficult.
- The accuracies of data are not known and in some cases of rigorous research, analysis may cause difficulties in validation and reliability estimation.
We will examine the sources of secondary data and their search first, and then discuss the collection of primary data.
Sources of secondary data
The sources of secondary data are sources internal to a firm or industry and sources that are external.
- The internal sources are:
- The external sources are:
- computerised data bases
- reports of associations
- reports and publications of government agencies
- other publications
Company accounting data These include accounting and financial details maintained in computers or ledgers; inventory, purchase, manufacturing activity logs; shop order files; worker files; payment ledger; cashbook; and so forth. When accessible, these provide data close to the primary data.
internal sources of secondary data
These constitute: Reports, accounting data, in house journal, computer database of the company and data warehouses.
Company reports These consist of annual reports and regular reports submitted to the board of directors or statutory reports submitted to the government. A great deal of routine data can be obtained through these reports on operational and performance aspects.
In-house journal Most corporations usually support in-house journals and useful general data may be obtained from them.
Miscellaneous reports These include consultancy reports; special reports of research supported by the firm, special reports to the top management in specific areas of management when new methods are introduced or new activities are taken up, or when the report is needed for trouble shooting or for large improvements in the firms’ operations.
Company computer databases The use of internal data for decision-making has increased so much in recent times that intranets (internal networks linking outward to the internet but forbidding access from outside) have begun to operate successfully. These are connected to the distributed organisational databases (Cooper & Schindler, 2000).
Enterprise data warehouses and data marts These have started coming up during the last 20 years, and seem to be gaining a competitive edge. The competitive edge seems to be coming more from using available information properly than by optimising approaches. A data warehouse is a collection of key pieces of information used to manage and direct business for the most profitable outcome and includes process managers who put the data into tables and the analysts make informal decisions from the tables (Anahory & Murray, 2000).
Public computer databases These large databases, which can be accessed with desktop computers, are fast growing and cover the areas of financial information, product sales and marketing channels, performances, manufacturing and inventory values, employment, and so forth. (They include bibliographic services too.) Accesses to these databases are available at a charge. Software with updating services are also available. For a treatment of data accessing through internet and searching for computer networks of secondary data refer to Cooper & Schindler, 2000).
external sources of secondary data
These constitute: Public computer databases, reports of industrial associations and syndicates, reports of government agencies and journals, computer databases and bibliographic services of professional and academic bodies.
Reports of associations Associations of industries like the Electrical Manufacturers Association, Automobile Association, and so forth have developed their own annual reports, which provide details of industry’s sales growth, operating characteristics, and the like. They also publish special reports and research statistics. Newspapers and magazines collect information from these associations and publish it. These are available on subscription.
Reports of government agencies Both central and state government departments provide large amounts of aggregated data and information on financial and operational activities and R&D activities in publications like the Reserve Bank of India (RBI) bulletin, details of industries in industrial classifications, census, industrial surveys and annual reports, economic surveys, and so on. The reports cover demographic details, housing, wages and income, production and sale of manufacturing and service organisations, agriculture and general employment. The relevant reports are surveys of current business, economic census, current industrial reports, census reports and the like.
Industrial syndicates These organisations also provide data on industrial services, plantwise/areawise information on manufactured products, inventories, sales and movement through marketing channels, financial ratios, and the like.
Other publications Other sources of useful data/information are academic publications such as books, professional journals, project reports, and dissertation abstracts. Computer search systems and computerised bibliographic services are also coming up fast in support of the individual researcher.
A sample list of secondary data sources is provided in Appendix B to this chapter, however, it is by no means exhaustive.
Computer Search for Secondary Data
There is an incredible potential source for research data in worldwide computer databases, electronic libraries and the internet (world wide web). For integrating the use of these sources into research efforts, a research scholar needs skills different from those required a decade ago.
computer search for secondary data
Specifying clearly the data needs and selecting an appropriate key-word, search can be made effective while accessing such computer data sources as electronic library, internet and electronic databases through a large number of search engines available.
The resources are not orderly arrays of files and documents and they cannot be considered as catalogues. Since a large quantum of information is changed and updated possibly every hour/every day, the researcher must be prepared for constant change. However, there are a number of ways to search these sources and find what is needed. Electronic searching sources also have to be used by the researcher along with electronic research sources.
The necessary information may be had from electronic databases, e-libraries, or the worldwide web. In finding and getting information or data for research purposes, it is desirable to follow some well planned procedures.
The following steps may be useful (Mckie, 1997).
- Specify your data needs.
- Select a keyword/search query.
- Select a suitable electronic library search source/database/search engine for the internet search.
- Save useful information resulting from the search.
- If necessary, repeat steps (2) and (3) with modified keywords.
Step 1—Specify information/data needs The problem of research should be defined as clearly as possible in order to be specific about the data/information needed. Since electronic searching requires ‘keyword’ use, the more specific/clear the research objective, the more suitable a keyword would be for close search.
Step 2—Select keywords A keyword is used for querying the database or web. Single keywords give very general information, which is not very useful. To be more efficient in search, use an appropriate ‘key phrase’.
Truncate the keyword Limit the data/information searched by specifying periods (1990–2000) or range (all people below a certain age) or type (technical manpower) or language (English) etc.
Use Boolean search (“AND”, “OR”, “NOT”) For example, instead of using keyword ‘scheduling semiconductor’, you may use “scheduling AND semiconductor” for searching for information on scheduling particularly in semiconductor manufacturing.
Step 3—Select a library database/website (search engine)/business database In selecting a particular website for particular data, the user has to look for scope of the data/information and the authority of the source of the data/information.
Library searches Many university and college libraries have catalogues that can be searched electronically through an internet connection. Hytel net (a telenet connection) is useful to gain access to remote library catalogue systems. Using tools like sociofile, current contents, and silver platter, it is possible to conduct fast catalogue searches. Data sets of secondary sources and questionnaires are available to students and faculty members of universities (in easy use form with SAS/SPSS packages). CD-ROM based products are also available in many countries for this purpose.
Web searches There are a number of search engines available for web searches. They differ in matters of display/presentation, options for keywords, and speed of updating. These deal with web addresses, names of sites, and names of documents useful in generalised searches. There are a large number of competent search engines that connect a large number of websites. A special site http://www.windweaver.com/index.htm is useful for knowing what type of search engine is useful for a particular search (http://www.Carleton.ca/~cmckie/research.htm is also useful). Some of the useful sites are given in Appendix B of this book.
The specific types of information searches are:
Known item searches ‘Where’ searches (databases or websites), ‘what’ searches, for example, the Google item.
Where searches: Lycos system gives geographical locations.
What searches: Alta Vista, Yahoo are useful.
Who searches: These provide information on people (all sites provide this information).
Some of the search engines are listed at the end of this section.
Step 4—Store information The useful information obtained in each cycle of search should be saved and stored suitably so that at the end of the search phase, the several bits can be organised systematically into a meaningful review of literature or a researchable data set.
A number of cycles of this search procedure (steps 2 to steps 4) may be necessary to obtain the required information for research.
Web addresses of a few search engine.
(For additional search engines and websites, see Appendix B).
From the above discussions, it is clear that larger amounts of secondary data are available at present to the researcher than ever before and at a much quicker time. The computer has brought most libraries and data sources around the world nearer to every researcher. To be efficient at using such secondary data, the researcher needs to devote additional skills and care at every step of research.
Primary data collection methods
Many methods and procedures have been developed to help in acquiring data. They make use of different means of describing and quantifying data. Each of these is suitable in specific situations of source and type of data. Too much dependence upon one method is not desirable. The recent trend in social science and management research is to use multiple methods. They serve two purposes; one of eliminating or minimising bias, the other of corroborating and, hence, providing greater reliability of data.
The primary data relevant to a problem is collected by one of the standard methods of research—experiments, surveys, field studies, case studies, system studies, and the like. The basic methods and tools for data collection for research include observation, questionnaire, interview, and projective technique.
The employment of any of these tools of data collection depends on the type of research that is being undertaken and the time and resources available to the researcher. Table 12.1 shows tools for data collection predominantly used in different types research.
Table 12.1 Tools for Data Collection Verses Types of Research
Observation is the most direct form of data collection. Usually data is gathered by observation when it can be gathered accurately only by this method. For example, behaviour of children (who cannot talk).
This is the most direct form of primary data collection. This is effective when data is accessible by observation of facial expression and body language relating to activities, which are repetitive and frequent. This should be supplemented by other methods as a check.
There is a considerable difference between casual observation and scientific observation. Scientific observation is well planned, recorded, and checked for validity and reliability. It has a research purpose and, therefore, has focus during its process (notwithstanding this, many haphazard observations have led to scientific discovery). Conditions under which data can be effectively collected are:
- Data must be accessible to the observer (private activity, motivation, and attitudes). Some behaviours are communicated through facial expression and body language. These are observable when the activity is repetitive, frequent, and predictable.
- They must be of short interval to reduce distortion due to recall.
- It is desirable that the observational data is used to supplement the other methods of data collection.
Sampling The subjects are sampled randomly and followed up (cf. QC - control samples are taken at equal intervals of time). Alternatively, a particular place or position is sampled randomly for occurrence of any activity (cf. work sampling, where particular activities are observed at random intervals). For an example, see Atkins (1978) in which parent-child interactions are observed.
Observational approaches can be classified as follows:
Basically there are two approaches. Systematic observation and participant observation. In the former recording schedule are used in direct or indirect observation and nonbehavioural and behavioural observations. In the latter insiders view point is obtained by the observer being embedded in the system, using analysis and collecting of data parallelly.
Natural verses contrived Observing how many cars pass a circle every hour is a natural observation, which is useful when phenomena occur frequently. A store worker checked by an observer disguised as a customer is a contrived observation.
Open verses disguised An observation made on television or metering certain occurrences are examples of open observations. Observations in lab experiments are often disguised, that is, the participants are unaware of certain kinds of observations made by the experimenter.
Structured verses unstructured In structured observation what aspects are to be observed and recorded are exactly known. The others are ignored; an observation checklist is usually used (cf. questionnaire). When any aspect of a phenomenon, as in exploratory situations, is considered useful, the observation becomes unstructured.
Direct verses indirect When current behaviour or occurrence is observed, the observation is direct. When past behaviour is to be observed, it can only be done through effects (physical traces). In this case, the observation becomes indirect. Direct observation is time consuming and costly, particularly if specific aspects or events have to be sampled (due to large times of waiting for the events to occur). Direct observation is not suitable for phenomena of long duration or rarely occurring phenomena. Data gathered through direct observation is not generally quantified (though it can be).
Mechanical verses human When devices like televisions, meters, video cameras, and photographic analysis are used for observation, it becomes mechanical. All other observations are human.
Non-behavioural and Behavioural Non-behavioural observations are analysis of records (historical or current records)—written, sound recorded or video taped, photographs or computer records, personal records. Content analysis may be used in this context, physical condition analysis, process and activity analysis, work, study, and manufacturing system analysis, and the like. Behaviour observation includes observation of (i) non-verbal behaviour, body movements, gestures, facial expressions; (ii) linguistic behaviour, which includes vocal sounds made during interactions; (iii) extra lingual behaviour like pitch of talking or manner of talking; and (iv) spatial behaviour, in which how an individual behaves in close physical proximity to others is studied.
Basically, there are two kinds of observation. The first is systematic observation, which has recording schedules; the second is participant observation, in which the observer is embedded in the system, and collection and analysis of data take place simultaneously (for details, refer the section on participant observation in the Chapter 6).
Evaluation of Observations as Data Collection Procedures
Structuring observations reduces (potential) bias of observation and increases reliability but reduced search in structured observations reduces validity. The observer must assimilate the information obtained in the observation and make inferences from them. It is at once a strength and weakness of behavioural observation. Good training is, therefore, necessary for the observer.
Disguised observation is often difficult and may cause loss of information obtainable in undisguised observation. Observations in contrived settings (lab settings) allow greater control over extraneous influences, affecting the results, but the experimental set up itself may introduce differences in the behaviour of the subject. Mechanical contrivances used in observation will free the observation from the observers’ selective process and bias but lacks the integrative and inferential powers of the human observer. Observation is not very useful for measuring attitudes, motivation, awareness, or knowledge. In such cases, the questionnaire mode is more appropriate.
Questionnaires are preferred in most surveys because they are less expensive. Skills required to administer them are also less than what is required for interviewing. Large samples can be dealt with simultaneously in questionnaires as mailing is possible. Further, the impersonal and standardised formats of questionnaires has uniformity (which is absent in interviewing) and tends to be a little more objective. The anonymity of the responder (particularly in mailed questionnaires) gives greater freedom of expression. However, if answers of certain questions are related to emotions and feelings, they may be better handled by a skillful interviewer than through a questionnaire. One great disadvantage is the low response rates (sometimes as low as 5 to 10 per cent).
This is the instrument for data collection in survey research for large samples. Impersonal and standard formats help in getting data objectively. Information on facts, attitudes, motivation and knowledge can be obtained easily.
Design of questionnaire A questionnaire is a formalised set of questions aimed at eliciting information regarding facts, level of knowledge, attitudes, needs, and motivations. At the current level of standardisation of the design of questionnaires, it can be treated more as an art than as a science. The cardinal approach to developing a questionnaire is to make it simple and avoid the inclusion of ambiguous and leading questions. The design of a questionnaire can be taken up in several steps. These steps are only considered as a convenient approach to design a good questionnaire rather than as a strict procedure for it.
Steps in the design of a questionnaire
steps in questionnaire design
There are many steps in this. Major ones are selecting the type of questions, selecting the methods and form of responses and pre-testing and revision of questions.
Step 1—Information sought The first step in any questionnaire design is the specification of the information sought. The exact information required has to be stated. Target population has to be specified. The objectives of the questionnaire have to be spelt out and the respondents should be so chosen as to make them less diversified. In this step, an attempt must be made to reduce both surrogate information error as well as respondent errors.
Step 2—Type of questionnaire The type of questions to be included in the questionnaire has to be decided. The major decisions are whether one or more questions need to be designed for each item of information. Getting a few quick trial responses and analysing the answers can facilitate this. Questions useful to the research problem only should be included. However, if additional questions are ‘thrown-in’, their purpose must be clear to the researcher. In general, questions may demand choice or preference by the respondent through an agree-disagree format.
Example Innovative tasks are mostly restricted to the R&D and Production Departments.
Two or more questions can be asked as one question. These are called ‘double-barreled’ questions.
Example The extent to which you are satisfied regarding the remuneration package and recruitment policies of the organisation, to enable it to recruit managers, are:
Demanding aggregations of several types of information should be avoided as far as possible.
The main issue involved in the design of questions is to specifically consider the respondent and his difficulties in answering. Some of the difficulties may arise because of the following reasons.
- Uninformed respondent Many simple questions are preferred to a single complex or composite question.
- Forgetful respondents When memory-based answers are required, aided recalls may be used to make the answers less error prone (errors are omission, creation, and telescoping).
- Inarticulate respondents When ‘why’ questions are used, the respondent may be unable to answer, though willing. In such situations, it is desirable to use projective techniques.
- Personal and embarrassing questions Preferably, either a random response method or counter-biasing statements should be used. In the random response method, the question is mixed with other questions and a single answer is elucidated.
Example A fairly unbiased percentage of rejection at the inspection stage can be obtained by asking the following questions.
- Acceptance rate at inspection for your product is _____ per cent.
- The acceptance rates at inspection for your company/s in general, in your sector is __________ per cent (100 - answer given in question (a)) = per cent rejection). Question (b) is inconsequential.
- Some respondents may be unwilling to answer and may deliberately distort the information. This can be verified by alternate questioning methods.
The content of the questions should be simple in meaning. Suggestive and leading questions introduce bias into the answers. When a question is designed, possible alternate answers must be kept in mind, particularly, if they are open ended. Assumptions must be clarified. The focus and reference of the questions should be such that the respondent tends to answer it objectively.
Example Clearly laid down policies are necessary for promoting innovation in industry.
- Policies and procedures for R&D in your organisation are:
- If you have answered the above question (i) as Ambiguous or Very Obscure to what extent, in your opinion, are absence of clearly defined policies the cause for the poor rate of innovation in your organisation.
Step 3—Method of administration When the questionnaire elicits information regarding behaviour—past and present—and intended attitudes and opinions, it is desirable to use an interview along with the questionnaire. Mail survey or telephone survey may be used, particularly for demographic properties like age, sex, income, occupation and the like, as in most survey research.
Step 4—Form of response The response format deals particularly with the freedom of the researcher and that of the respondent. The formats usually used are open-ended and multiple-choice questions.
- In the open-ended format, the respondent is free to give any reply. A wide range of answers is obtained in this type of question, that is, variance of the relevant variable tends to be rather high. But in certain cases, some special answers may be very useful. The researcher in no way influences the respondent for a particular type of answer. Errors due to lack of articulation are present and may decrease with probing.
- Multiple choice questions are easier to answer and reduce interviewer bias. Gradations of answers are possible, but involve greater design effort on the part of the researcher. Answers, other than those offered, are lost. Usually, the problems will be regarding the number of alternatives to give and to check whether the alternatives are balanced or unbalanced. Balanced designs are universally preferred. To decide which method is better, a split ballot approach can be used. Two or more versions of the questions are offered to a small group of respondents. The best one is chosen.
Step 5—Wording and phrasing of questions In this regard, the basic principle is to make the questions simple and straightforward. Increasing the length of the questions for the same answer is desirable.
Example Capital productivity (where highly machine-oriented operations are involved), material productivity (when cost of material is a major portion of product cost) and labour productivity (when operations are mostly manual) are a few productivity indices used in manufacturing organisations.
Considering the most relevant one for your organisation, how would you rate your organisation’s productivity?
Step 6—Sequence of questions The sequence of questions may be as follows:
- Simple questions in the beginning to complex questions in the end, with increasing complexity.
- Questions must be free from bias in the beginning.
- Neutral questions in the beginning, sensitive ones in the middle, and controversial ones in the end are desirable because generally it has been found that the efficiency of response is high in the beginning and low at the end.
Step 7—Layout of the questions The format and layout of the questionnaire should be physically designed so as to eliminate recording errors. Branching questions (like in computer programmes) should be avoided, particularly on mailed questionnaires. Sometimes, it is desirable to include codes with the questions.
Step 8—Iteration Like in any design function, it may be necessary to go through steps one to seven in one or two iterations to eliminate undesirable features of questions.
Step 9—Pre-testing and revision of questions The pre-test is generally done on a small sample of respondents who are similar to the respondents of the main study (undergraduates in campus in place of senior executives is to be avoided!). The main questions involved in pre-test are the following:
- Which are the items to be pre-tested? The guiding factor is to choose questions, in which the interviewer’s errors tends to be large.
- How is the pre-test conducted? It is essential to interview the respondents after answering the questionnaire and ascertain why the questions were answered the way they were. Alternatively, protocol analysing (loud thinking of the respondent during answering), and debriefing may also be used.
- Is the number of respondents sufficient? Adequate sample size should be determined for the pre-test (cf. Sample size determination).
Interviews can be used in all segments of the population, whereas questionnaires can be used only with educated segments, particularly where the questions tend to become complex. The sample for interviews tends to be more representative than in the case of questionnaires as people are generally more inclined to talk than to write. Interviews are flexible and in this sense a little more customer-oriented than standard, which is the case with questionnaires. Questions not understood by respondents can be explained, which is not possible with questionnaires. A skillful interviewer can overcome unwillingness on the part of the respondent to answer emotionally laden or complex questions.
Interviews can be used for data collection for all segments of population whereas questionnaire can be used only with the educated segment. They are the commenest method to get information on behaviour, attitudes, needs and characteristics of people. Interviews are in-depth in nature.
Depth interview Interviews are the most common methods of obtaining information on behaviour, attitudes, needs, and characteristics of people. One method of getting information is to ask directly. But direct questions may fail to elicit proper answers because the subject is unwilling, or the questions are embarrassing, or for other reasons. Depth interview is a technique used under these circumstances. There are two kinds of depth interviews—individual depth interview and focused group interview. Focused group interviews are most widely used in management research.
Individual depth interview This may take anywhere between 1/2 an hour to 45 minutes. The interviewer asks no specific set of questions. There is freedom for the interviewer to create questions as he goes along with the interview. On the other hand, the respondent has the freedom of response, both with respect to content and with respect to manner. The interviewer should not try to influence the respondent in any way to change, either the content or the form. Ideas obtained through individual depth interview are generally of high quality (Fern, 1982). Individual depth interview is used:
- When detailed probing is necessary to elicit answers on behaviour, attitude, character, and needs.
- When the subject matter is confidential.
- When emotions or embarrassment may be evoked while answering the questions.
- When obtaining answers to questions which would be constrained in groups because of the subject’s conformity to socially acceptable norms.
- When information on complex behaviour patterns is desired while interviewing professionals.
- When it may be desirable to allow the respondent to sketch anecdotes or tell stories.
individual depth interview
The interviewer develops freely questions as he or she goes along the interviewing process and restrains from influencing the respondent who is free to respond as he or she likes.
Fern (1982) investigated into the assumptions of focus group for idea generation and obtained a few useful results.
Individual interviews produce higher quality and higher volume of ideas per individual, which are useful in exploratory research. One of the major disadvantages of individual depth interview is that it is tiring and only a small number of people can be interviewed per day, say five to six persons.
Focused group interview A group of 8 to 12 individuals is formed to reflect a particular behaviour or characteristic and is led by a moderator. The moderator must develop a rapport with the group, set structured group rules for interaction, and clarify objectives of the interview. He must evoke intense discussion. Finally, he must summarise the outcome of the interview.
focused group interview
A group of individuals is interviewed by the researcher focusing on a given experience and its effects. Group is led by a moderator and is informed of topic in advance. Respondents are free to express their own lines of thought but are controlled only to the general direction of the responses to the subject under focus.
The main function of the interviewer is to focus attention upon a given experience and its effects. What topics and what aspects of the topics are to be covered is known in advance to the interviewer. The topics or their aspects are derived from the research problem by the interviewer and/or based on the experience in which the respondent has participated, or both. The list of topics or aspects forms a framework for the interview but the manner in which the questions are framed and asked is left largely to the interviewer’s discretion. The respondent is free to express his own line of thought; however, the interviewer controls the direction of the interview to achieve focus by confirming or guiding the respondents to give him the information he wants.
The persons interviewed are known to have been involved in the particular situation and the interviewer has analysed the theoretically significant elements, patterns, and structure of the situation. Also, the scientist has arrived at certain propositions or hypotheses regarding the situation. Based on these two premises, he develops a guide to the interview, setting forth the major areas of inquiry and the kind of data to be obtained. This enables the interviewer to get the situation defined as per the respondent group and modify or sharpen his hypothesis or proposition. A more pervasive definition of focused interview could include any interview in which the interviewer knows in advance what specific aspects of an experience he wishes to have the respondent group cover, whether or not the interviewer has observed or analysed the specific situation.
A focussed group has several advantages. Individuals can sharpen their opinions. More accurate information can be obtained by the focus group interview than by interviewing a single individual. Security in belonging to a crowd urges some inarticulate respondents to speak out. The answers in a focus group interview are more spontaneous and feel for the area represented by the group is much greater. Volume of ideas per individual may be smaller and the cost of interviewing will be lower for focus interviews. The determination of correct group sizes is very important for focus groups. Two groups of four persons each may help in getting more information than a single eight-person group. Its disadvantages are that it is difficult to ascertain a random sample for focussed group. Participants may play games, distorting the outcome of the interview. Strong individuals affect group reactions and moderator biases are present. Also, generalisation from group to population is generally risky. [Refer to chapter by Alan Hedgs in Walker, 1985.]
The underlying theory behind projective techniques is that the description of vague and fuzzy objects requires interpretations based on one’s experience, attitudes, and values. The more vague the object is, the greater the revelation of the respondent. There are four types of projective techniques: (i) Association tests, (ii) Completion tests, (iii) Construction techniques, and (iv) Expression techniques. All have originated from clinical psychology.
These require a well trained interviewer. In this form of interview evaluations are obtained on vague and fuzzy objects or pictures and interpretations are made by the interviewer to measure attitudes feelings and experiences.
Association tests In these tests either the first thought or word that comes to mind when the researcher presents a word or phrase (free word association) or successive words or thoughts that come to mind (successive word organisation), is given by the respondent. The responses are analysed for measuring attitudes to certain category of stimuli.
Completion tests In sentence completion tests, the respondent completes an incomplete sentence with the phrase or word, that comes first to his mind (cf. open-ended question). In strong completion techniques, which are an extended version of the S.C.T., an incomplete sentence is completed based on the respondents’ experiences and attitudes.
Construction techniques The respondent is asked to construct (that is, develop) a stray diagram or a description by just presenting a visual depiction of a context.
Cartoon techniques A cartoon or a drawing is shown to the respondent and the respondent writes a story or draws a picture. This will be the projection of the sub-conscious mind or socially unacceptable feelings.
Third person techniques The respondent is informed about what a third person possesses or does or which characteristic he has. The respondent is then asked to express his feelings towards these.
Vague pictures Details are as above.
Expression techniques This is mainly role-playing (by the respondents) of the behaviour of another person.
All these techniques require a highly trained interviewer. They tend to be expensive and since the samples are small and are non-probable in nature, the respondents are generally non-representative. Considerable measurement errors creep in. It has one advantage, however, that is, of uncovering aspects not given out by direct interviews. Often, this helps in hypothesis generation. (Anastasi, 1982.)
Sampling errors are statistical errors arising out of drawing inferences from sample to population. They can be estimated. It is obvious that in a complete enumeration census, sampling errors will be totally absent. But the data gathered will still have errors due to other reasons. These errors are called non-sampling errors. Non-sampling errors are common to both research sample surveys and complete enumerative surveys. Errors arise in the method of data collection, data processing, and in the responding elements. The following paragraphs briefly discuss the non-sampling errors related to sample surveys.
These errors relate to the data collection procedures of observation interview and survey. They are of two kinds. Non-observation errors which are due to some elements not having been studied and observation errors which are due to respondents not answering certain questions / answering deliberately wrongly.
The sources of errors are (Murthy, 1967):
- Inadequate data specification.
- Inadequate or faulty methods of interview, observation, or measurement.
- Lack of trained investigators.
- Inadequate scrutiny of basic data.
- Errors in coding data entry and verification.
- Errors in presentation and printing.
- Non-coverage errors.
- Response errors.
These however could be treated as two errors, non-observation errors and observation errors.
These errors arise in sampling surveys because some population elements of interest are not studied. Non-inclusion may be due to an inadequate sampling frame in which some elements of interest are left out, either as a result of wrong procedure (incomplete listing) or deliberately, because of practical difficulties. These are termed non-coverage errors. Sometimes there may be over coverage, that is, the same items listed differently in different lists may get duplicated. For example, when a company manufacturing multiple family products is listed under different product classifications and product classification lists are used as frame, the same organisation may get duplicated in the sample.
The other major types of non-sampling errors are non-response errors. These arise because the subjects included in the sample did not respond. The non-response may be straightforward refusal to respond to the survey, delaying the response testing, or the persistence of the researcher to a point beyond acceptable time limit.
These are field errors caused by the respondent not answering specific questions; or answering certain questions deliberately wrongly, ambiguously, or carelessly, which render the questionnaire one of doubtful utility. Interviews are much better than questionnaires from the point of view of observation errors.
Handling non-sampling errors Improving the sampling frame and adjusting the results appropriately can reduce non-coverage errors. Non-response errors can be reduced by making prior appointments, by follow-ups, and by convincing the respondent of the importance and value of his response to the research (for details see Churchill, 1987, Chapter 11).
Validity and reliability of data collection procedures
Validity and reliability in data collection relates not only to accuracy in the data items that are measured but its accuracy with respect to the purpose for which it was collected.
This validity for a data collection procedure can be obtained by subjecting the data to confirmatory factor analysis to confirm whether theorized dimensions emerge from the data.
Factorial validity can be established by submitting the data to confirmatory factor analysis. The results of factor analysis (a multivariate technique) will confirm whether or not the theorised dimensions emerge. The measures are developed by first delineating the dimensions, so as to operationalise the concept. Factor analysis would reveal whether the theorised dimensions are indeed tapped by the items in the measure. Criterion-related validity can be established by methods, that are already discussed in section on validity is measurement in chapter 10.
The reliability of a measure is established by testing for both consistency and stability. Consistency indicates how the items measuring a concept hang well together as a set. Cronbach’s alpha is a reliability coefficient that reflects how well the items in a set are positively correlated to one another. Cronbach’s alpha is computed in terms of the average intercorrelations among the items measuring the concept. The closer Cronbach’s alpha is to 1, the higher the internal consistency reliability (Kerlinger, 1986).
Split half reliability coefficient is obtained by splitting the total questions set into two half sets and correlating their scores. This tests the consistency of data collection procedure.
The methods to be followed for consistency, reliability and stability, as discussed in section on reliability in measurement in chapter 10, can also be applied to data collection procedures. The following sections discuss some additional aspects of validity and reliability of interviews, observation, and questionnaires.
Another measure of consistency (reliability) used in specific situations is the split half reliability coefficient. Since this reflects the correlations between two halves of a set of items, the coefficients obtained will vary depending on how the scale is split. Sometimes split-half reliability is obtained to test for consistency when more than one scale, dimension, or factor is assessed, and the items across each of the dimensions or factors are split based on some predetermined logic (Campbell, 1976). In almost every case, Cronbach’s alpha is an adequate test of internal consistency/reliability.
As discussed earlier, the stability of a measure can be assessed through parallel form reliability and test-retest reliability. When a high correlation between two similar forms of a measure is obtained, parallel form reliability is established. Test-retest reliability can be established by computing the correlation between the same tests administered at two different time periods (Sekaran, 2000).
Validity and Reliability of Interviews
The key to an effective interview is establishing rapport. The relative status of the interviewer and his ethnic origin facilitate the establishment of rapport. Women and younger people generally seem to be more successful in eliciting information (Best and Kahn, 1986). Experience tends to increase interviewing skills. Interviews based on carefully designed structures are generally more valid. Critical judgement of experts is helpful in determining the questions. Restating questions in an altered form will ensure reliability or repeating the interview would also help in this. Danger of interview bias is constant, however, inexperienced researchers do not ordinarily possess bias.
Validity and Reliability of Observation
In order that observation is valid to a satisfactory degree, significant incident of behaviour must be identified. One way to ensure this is to supplement the knowledge and skill of the researcher with the judgement of experts in the field and the relationship of qualities in the incidents to establish theories. Researchers may tend to see what they expect to see and to overlook incidents that do not fit their theory (Best and Kahn, 1986). This may distort their observation. It is better to use independent observers trained in observing and recording. Observation try-outs may be used in the initial phase. Joint observation helps build reliability. Randomly selected time samples (5 seconds or 10 seconds) may help more in obtaining unbiased and representative samples of behaviour and would enhance both validity and reliability.
Validity and Reliability of Questionnaires
A possible procedure for validation and ensuring reliability of questionnaires is suggested below:
- Ask the right questions.
- Phrase them in the least ambiguous manner.
- Clearly define the important terms.
- Rate the content validity.
- Obtain predictive validation by correlating questionnaire scores with observed behaviour or peer/superior ratings.
- Administer the questionnaire to the same participants twice and compare the two responses.
- Get the first four listed above checked by an expert.
The two sources of data for a research problem are secondary data, which was already collected for some other purpose but useful for solving the research problem; and primary data, which has to be specially collected by the researcher through the use of tools of data collection. Secondary data can be internal to a company, like company reports and in-house journals, or external to it, like computer databases and government publications. The advantage of secondary data is the low cost and effort to obtain data. The researcher—using one or more tools like observation, questionnaire, interview and projective techniques—gathers primary data. The type of tool used depends on research design, but generally a combination of two or more needs to be used. Observation is the most direct method of data collection and is used when other methods are not feasible. Facial expression and body language can give to the observer rich information.
In most surveys, questionnaires are preferred because skills required for data collection are less, large samples can be handled easily and there is greater objectivity in the method. But the main limitation is low response rates. The sine qua non of questionnaire design is to keep it simple, unambiguous, and to carefully sequence the questions.
Interviews are best suited for obtaining information on attitudes, behaviour, needs and characteristics of people. Depth interviews by a trained and skillful interviewer are best suited for situations where information on emotionally-laden issues are being probed. Focused interviews are group interviews, that elicit information from a group on its experience in a specific problem area or situation. Projective techniques like completion test and cartoon techniques are intended to uncover subconscious feelings and opinions of the responder, which no other technique can provide.
- Bernard, Russel H. (2000). Social Research Methods. London: Sage.
- Cartwright, C. A. and G.P. Cartwright (1974). Developing Observational Skills. New York: McGraw-Hill.
- Peterson, Robert A. (2000). Constructing Effective Questionnaires. New York: Barnes and Noble.
- Walker, R. (1985). Applied Qualitative Research. Aldershot: Gower.
- Zimmerman, Donald and Muraski Michail Lynn (1995). Elements of Information Gathering: A Guide for Technical Communicators, Scientists and Engineers. Phoenix: Oryx Press.
1. What is the difference between (i) natural and contrived observation, and (ii) direct and indirect observation? Explain with examples.
2. When do you use the following?
- Depth interview schedule
- A mailed questionnaire
- Focused interview
- An unstructured interview
3. What is secondary data? When do you use it for research? Are there different types of secondary data? If yes, give examples.
4. What are the choices of tools available for collecting primary data? Give examples.
5. Study a few research reports or dissertations that use questionnaires. What types of questions were used? Was the choice of the question type adequate for the research undertaken? Comment.
6. Which type of data collection method would you use for the following situations? Justify your choice.
- You are required to find out what kind of leadership exists in a large organisation.
- There is a general feeling among researchers that technology development effort in an industrial scenario, like the Indian one, dominated by foreign collaborations, is low key. You have to verify it.
- You have to assess what kind of training programmes are popular with industrial organisations.
- Introduction of total quality management approaches are believed to be ineffective in some organisations. You have to find out whether it is a fact and if so why.
- In implementing modern management techniques the research question is CEOs acceptance is less important than the acceptance of the lower level managers and workers.
7. Search for a few good field survey research studies. Get the details of the research framework and variables. Develop questionnaires/interview schedules for data collection. Check against the one actually used or discuss the merits of your questionnaire. Test it against a small sample and note its shortcomings.
8. How are tests different from questionnaires? When do you use tests and when do you generally design a questionnaire? Give some examples.
9. Choose a questionnaire from a thesis. Carefully study (i) the type of scales, (ii) the working of questions, (iii) the layout of questions, (iv) form of response. Do you agree with each for the research continued in the thesis? Comment in detail.
10. How do you reduce non-sampling errors during data collection in (i) questionnaires and (ii) interviews?
11. What is question layout? Why is careful consideration of layout important in questionnaire construction?
12. In what research situations are projective techniques useful? What are the disadvantages of projective techniques?
13. If repeatedly gathered data shows the same results over a number of replications, are the results valid? Or are they reliable, or both?
14. What would you do to validate and verify the reliability of a questionnaire when it consists of a mix of homogeneous and non-homogeneous questions? Explain your approach.
15. What would you do to validate a questionnaire that has only questions demanding yes/no responses?
16. If you have validated a questionnaire, is it necessary to validate the responses (data) obtained through it? Discuss. If your answer is yes, fully/partially, explain how would you do it?
17. Evaluate the following questionnaire portions from the point of view of format and wording of questions.
(A) Questionnaire Set (Mukund Mohan, 1986)
Instructions: The questionnaire set is a part of a study undertaken on implementation of production/operations management techniques in selected Indian industries. Your company is included in the sample required for the study.
Every item begins with a Statement. Relate the Statement with the project under study, and then answer Query 1. Only if, the answer is ‘Yes’, proceed further, otherwise move onto the next item. It is important to keep in mind “For the project under study - -” while you answer Query 2. Answer the Query 3 in consonance with the Query 2.
Please note the following points, while you answer:
- Terms such as Adequacy, Efficiency, Sufficiency, Appropriateness, and Effectiveness should be taken only in a subjective sense.
- Do not consult anybody. Your cooperation is essential for the successful completion of the study.
Statement: The structuredness of a problem situation, in the sense that a standard or readymade solution or model is available for implementation and problem solving, is an important factor for successful implementation.
Do you agree with the statement? Agree__________Disagree__________
For the project under study, the degree of structuredness of the problem-situation was?__________
The structuredness of the problem situation was to the extent of__________
Statement: Participation of workers and other supervisory personnel in the design and implementation phases is an important factor for successful implementation.
Do you agree with the statement?
For the project under study, the degree of participation was__________
The participation was to the extent of ________
Statement: The modernity of the technology for production compared to the current state of the technology is an important factor for successful implementation.
Do you agree with the statement?
For the project under study, a degree of modernity of the technology was__________
The modernity of the technology was to the extent of __________
Statement: The objective of problem solving and implementation should be well determined for successful implementation.
Do you agree with the statement?
For the project under study, a the degree of clarity of the objective was__________
The clarity of objective was to the extent of __________
Statement: Resistance from shop-floor personnel should be sufficiently resolved for successful implementation.
Do you agree with the statement?
For the project under study, the degree of sufficiency of resistance resolution was__________
The sufficiency of the resistance resolution was to the extent __________
Statement: Changes within the organisation should occur as per the plan, within a specified period, for successful implementation.
Do you agree with the statement?
For the project under study, the degree of implementation success was__________
The implementation success was to the extent of __________
Attachment to the firm
If you had your way, how many years would you remain with this company?
Overall job satisfaction
I rate my division/department higher, as a place to work than other organisations I know about.
How would you describe the leadership climate in your Division?
Satisfaction about leadership climate
Do you find superiors willing to make time to discuss a problem that might be worrying you?
Taking everything into account (the pay, the people you work with, supervision and the work you do) how do you feel about your job?
Given an opportunity to be the leader of a prestigious project of the organisation, what would you do?
Would you recommend your son/daughter to join this organisation?
How would you like yourself to be identified as?
What was the last major development in the organisation?
How would you describe your identity in the organisation?
Please put these 5 things in order of importance.