Planning Stage

	Stakeholder Survey Survey Questions: I would like to begin the survey by asking in what areas within the arts do you have an interest? What is your interest and why do you have that interest? How did that interest develop? Have you been involved in the arts either as a participant, administrator, or an audience member? [Participant? Administrator? Audience Member?] How and of what use do you think the arts are to quality of life? Can you think of something that the arts should be providing or achieving that is not currently happening? Can you tell me in your own words, how you define "the arts"? What background have you had in the arts? Have you had any training in the arts? [Yes, No; If "yes," then what?] Are you aware of any publicly funded statewide organizations that provide support to individual artists as well as arts organizations? [Yes, No; If "yes," whom or what?] As I mentioned previously, the Ohio Arts Council is working to develop a study which will address issues related to the Arts in Ohio. What would you like to see, learn about, or find out regarding the arts in Ohio and why? Can you think of anything else that you feel would be useful in a report examining the Arts in Ohio. [Continue probing until nothing else to offer] Can you think of anything that others might find useful that you haven't mentioned? [Why, if not apparent?] Now, keeping in mind that this is part of a planning process for developing a State of the Arts report, I would like to ask you about information that you would want to know related to some specific areas. What would you want to know about the arts with respect to quality of life? How about in respect to defining one's identity within a specific group, culture, or community? (What would you want to know about the arts with respect to defining one's identity within a specific group, culture, or community). How about in regard to economic growth? (What would you want to know about the arts with respect to economic growth)? How about in regard to an educated and aware citizenry? And how about in regard to enhancing one's creativity and/or spirituality? Now in a much more general sense, not specifically related to the arts, as an individual what issues concern you about the future? In your capacity as [title] what issues concern you about the future? Do you feel there is any way that a state of the arts report could address or impact these concerns? [Yes, No; if "yes," how?] And lastly, can you tell me what you think will be the most significant policy issues in the arts over the next five years? In June 1998, the Ohio Arts Council (OAC) approached the Center for Survey Research concerning its need to gather data that would assist in developing a research project for a State of the Arts Report. The OAC was concerned that the larger State of the Arts project comprehensively cover the interests of the various constituencies that, ultimately, would be the consumers of the report. Although a typical method for gathering such preliminary data is through focus groups, issues in this project, such as diversity of the constituents, geography, and fiscal prudence, served as dissuasive factors for employing this approach. Following discussion, the Center and the OAC decided that a telephone interview process would likely produce the desired information, provided the interviewing staff was retrained to collect the information. To successfully gather the desired information, interviewers were trained to deviate from the established protocol of adhering strictly to a script. The current project required a more conversational and interactive approach, similar to what respondents would experience within a focus group. The questions were designed to be open and vague, with the intention of eliciting the original, novel thoughts of respondents. An example of this is, "Can you tell me in your own words how you define ‘the arts'?" Interviewers were instructed to ask questions as worded, but were told they could incorporate information provided by the respondent in previous answers. This was considered critical to maintaining the engagement of the respondent who might otherwise feel the interviewer was disinterested. Acknowledging information previously provided by respondents serves to reinsure them that what they are providing is important. An illustration of this can be seen through an examination of the first three questions: Question One: I would like to begin the survey by asking in what areas within the arts do you have an interest? Many respondents listed multiple areas with varying level of detail. Often this detail included the area(s) of interest (I really enjoy music…), how the interest developed (and have ever since I was introduced to the trumpet in grade school), and the degree of involvement (I played throughout primary and secondary school and then joined the marching band in college; I still play occasionally with the alumni organization). Interviewers followed such responses with additional benign probes, such as "Can you think of any other areas in which you have an interest?" Question Two: What is your interest and why do you have that interest? Obviously, if the interviewer had received the above response and strictly adhered to the script, the respondent would assume the interviewer wasn't listening and would likely terminate the interview or invest less in the thoughtfulness of his or her responses. As a result, interviewers were encouraged to proceed with this question by first acknowledging the relevancy of what was previously said and then probing for additional information relevant to the question. If a respondent had been more forthcoming about how one interest had developed than another that he or she mentioned, then the interviewer focused on the one that lacked sufficient detail. Essentially, the interviewer worked to "fill the holes" and explore any additional information that may have been neglected. Question Three: How did that interest develop? Again, the interviewer would preface the question with the previously acquired information and explore anything that might have been neglected. (Now I understand that your interest in music began when you were introduced to the trumpet in grade school. Can you think of anything else that served to encourage this interest?) Interviewers were also instructed to paraphrase and repeat the respondents' comments back to them to ensure a complete and accurate understanding of the information provided. In addition, interviewers were trained to avoid certain situations, such as engaging in conversation that deviated significantly from the question or topic at hand. Such deviations could lengthen the interview process as well as precipitate the introduction of bias. Interviewers were also cautioned against giving respondents ideas about how to answer (such as "Have you ever been involved in the arts as a participant, like having played in a band?") or indicating how other respondents answered the question. Additionally, interviewers were taught how to best utilize the information provided while minimizing the possibility that such information would introduce or create bias. Like most telephone survey organizations, the Center uses the Computer-Assisted Survey Execution System (CASES) almost exclusively in its telephone survey data collection projects. However, while such programs facilitate data management and processing, they can slow the interview process and distract interviewers through concentrating more on negotiating the program than interacting with respondents. As a result, it was decided early on that this project would be completed using a paper-and-pencil format. This allowed interviewers to take notes and review points during the interviewing process, as well as to update answers to previous questions when new information was provided later in the interview. The OAC determined 16 diverse groups from which to obtain information for this project and provided lists of membership and contact information for each of these groups to the Center. The Center then determined the number of entries for these lists and, using SPSS, generated a list of random numbers. The Center used this information to generate callsheets for the 16 groups. All 16 samples were managed individually to ensure that 10 or more completed interviews were conducted for each group. Multiple calls were made and appointments were scheduled as necessary to obtain the involvement of selected respondents. Since many of the respondents (i.e., legislators and government officials) screen telephone calls, an overview of the interview was prepared on letterhead and provided upon request to prospective respondents. There were 186 interviews completed with only nine prospective respondents refusing involvement (or 5%). Although this project was designed more qualitatively than quantitatively with a focus on the collection of ideas to delineate the more salient aspects of what should be covered in a future State of the Arts report, some OAC committee members expressed an interest in having all data coded and analyzed to assist them in examining the data for trends. As a result, data for all 24 questions were coded by different sets of coding staff and entered into a database. From this, inter-rater reliabilities were established for all questions separately and for the instrument as a whole. Inter-rater reliabilities varied from a low of 44% to a high of 96%, with the inter-rater reliability for the entire instrument (all 24 coded items) being 68%. Given the extreme open-ended nature of the items, these reliabilities were considered quite positive. In addition, much of the variation was found to be the result of coding categories with significant overlap and responses that one coder would give a specific category assignment while the second coder felt sufficient ambiguity existed to warrant an assignment to the "other" category which could then be flagged for further review. Literature Review