National Academies Press: OpenBook

Evaluation of the Department of Veterans Affairs Mental Health Services (2018)

Chapter: Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes

« Previous: Appendix A: Supporting Documentation for the Survey
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×

Appendix B

Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes
*

IOM1 VA MH Services Evaluation

VA Staff

Discussion Protocol
for Individual Interviews and Small-Group Discussions

Thank you for agreeing to participate in this discussion today. My name is ________________ and this is my colleague ________________. We work for Westat, a research organization based in Rockville, MD. Westat is under contract to the Institute of Medicine (IOM), part of the National Academy of Sciences, to undertake a Congressionally mandated study of the array of mental health services available to OIF/OEF/OND veterans through the VA (for example, individual or group therapy, substance abuse treatment, etc.), and to focus on why some of these veterans are not using the VA services. Maybe they have used the services in the past and stopped, or maybe they have never come to the VA for assistance. Also attending today’s discussion is ________________ [IOM committee member] and ________________ [IOM staff member]. Today we’re interested in hearing your perspectives on these issues. We want to learn what subgroups of this younger cohort of veterans you believe the VA is able to reach and serve well; which OIF/OEF/OND veterans are more difficult to engage in mental health services and why you think that may be; and we want to learn about any outreach strategies your VAMC/network is taking—or you think could take—to better engage this hard-to-reach population.

___________________

* The following documents were prepared by Westat, an independent research corporation, which assisted the committee with the design, implementation, and analysis of the site visits.

1 At the time the survey work began, the Institute of Medicine was a program unit in the National Academies of Sciences, Engineering, and Medicine. After an organizational restructure in March 2016, the Health and Medicine Division of the National Academies carries out the work previously undertaken by the Institute of Medicine.

Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×

Before we get started there are a few things I should mention. This is a research project. Your participation is voluntary, refusal to participate will involve no penalty or loss of benefits to which you are otherwise entitled, and you may discontinue participation at any time without penalty or loss of benefits to which you are otherwise entitled. If you choose to participate, you don’t have to answer any of our questions that make you uncomfortable. We have planned for this discussion to last no more than 60 minutes.

We will be going to all 21 VISNs to see if there are common issues across geographic areas, or if there are strategies being successfully implemented in some locations that could be tried in other venues. After each visit, we will submit a brief, high level report to the VA that summarizes the major findings from the visit. This report will be submitted to the IOM’s public access file for the study. We will also submit reports summarizing our findings across multiple sites to a committee that has been assembled through the Institute of Medicine; the committee will then incorporate our findings into a larger, overall report that will go to the Department of Veterans Affairs and the Congress. [If an IOM committee member is attending the visit, then note that s/he also will prepare a brief report that will be submitted to the IOM’s public access file for the study.] The final report from this study is scheduled to be released to the public in mid 2017. However, your name will NOT be used in any of the reports. We aim to summarize findings such that comments cannot be attributed to a particular individual. No personally identifiable information will be shared with anyone outside of the site visit team here today. I would ask that you respect this privacy goal and that whatever is said in this room among you, stays in this room.

To further help us protect your privacy, we have obtained a certificate of confidentiality from the U.S. Department of Health and Human Services. With this certificate we cannot be forced (for example, by court order or subpoena) to disclose information that may identify you in any federal, state, local, civil, criminal, legislative, administrative, or other proceedings. The researchers will use the certificate to resist any demands for information that would identify you, except to prevent serious harm to you or others, and as explained below. You should understand that a certificate of confidentiality does not prevent you or a member of your family from voluntarily releasing information about yourself, your family, or your involvement in this study. If an insurer or employer learns about your participation, and obtains your consent to receive research information, then we may not use the certificate of confidentiality to withhold this information. This means that you and your family must also actively protect your own privacy. A certificate of confidentiality does not represent an endorsement of the research study by the Department of Health and Human Services or the National Institutes of Health. The only time we would need to break confidentiality is if we heard that someone was planning to harm him/herself or someone else.

Do you have any questions? [ANSWER ALL QUESTIONS]

Finally, with your permission, we would like to record this discussion. This recording will be used to help us recall exactly what was said when we go to summarize our findings. The recordings and any notes we have will be stored on Westat’s server. They will be accessible only to the Westat project team. We will destroy the recordings after the study is complete. Are you okay with us recording?

IF PERMISSION WAS GIVEN TO RECORD ASK AGAIN IF THERE ARE ANY QUESTIONS. ANSWER ALL QUESTIONS.

Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×

If there are no further questions or concerns, I’d like to start the audio recording now.

TURN ON RECORDER: For the purposes of the recording I am going to ask each of you to state out loud if you are willing to participate in the discussion and if I have your permission to audio tape. GO AROUND THE ROOM AND ALLOW EACH PARTICIPANT TO STATE HIS/HER AGREEMENT TO PARTICIPATE AND BE AUDIO-RECORDED.

I. INTRODUCTIONS

I’d like to start by having each of you introduce yourselves. Please tell us your first name only, what department you work in and your area of training (for example, “I’m a psychiatrist, but I work in primary care”), how long you’ve been there, and, briefly, what role you have in providing mental health services to OIF/OEF/OND veterans.

II. ACCESS

  1. What activities or strategies undertaken by [this VA facility] do you believe to be most effective at engaging OIF/OEF/OND veterans in mental health services? PROBE:
    • What venues are most effective (e.g., health fairs, college campuses)?
    • Particular partnerships that work well?
    • Specific materials or messaging strategies that seem to reach this population the best?
  2. How well do these strategies engage women veterans? Minority veterans? What, if any, population-specific strategies have been tried that have been successful?
  3. What, if anything, have you tried that has turned out to be completely unsuccessful? That is, you thought it would be a reasonable approach, but it turned out to be a total flop. Why do you think that approach was not successful?

III. QUALITY

  1. Which mental health services that you offer to this population are most effective and why? What does “effective” look like? For example, how do you know the veteran is getting better? [PROBE: Decrease in symptoms? Increased ability to live with symptoms? Able to do things with treatment that was not able to do when s/he came in for help?]
  2. What services do you wish you could offer, but you are not able to within this facility?
    • What would be needed in order for your facility to provide these services?
    • Is there anywhere you can refer veterans to in the community who does provide this?
  3. What sources of information does [this VA facility] use when it undertakes a quality improvement effort? Please give an example of one such effort, if possible.
  4. What is the process by which staff can make recommendations for improvements to services? What, if any, recommendations have staff at this site made? What changes, if any, have come about as a result of those recommendations?
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×

IV. CHOICES

  1. Of the mental health services provided by [this VA facility], which ones do OIF/OEF/OND veterans seem to like the best? Why do you think that is?
  2. What options are available to veterans if they are not happy with their clinicians or the services they are receiving?
  3. What kinds of health technologies is this facility using to support this younger population of veterans? [IF NEEDED: By health technology, we are thinking of such things as phone apps, videotelehealth services, or web-based tools.] PROBE:
    • Which of these services are most popular with the OIF/OEF/OND veterans?
    • Are there other services this facility is considering offering? Which ones and why?

V. BARRIERS

  1. The underlying goal of this study is to learn more about those veterans of the wars in Iraq and Afghanistan who are not using any of the services you all have described. Why do you think some OIF/OEF/OND veterans are not coming to [this VA facility] for mental health services? [PROBE: Transportation? Stigma? Not aware of available services?]
  2. Are there specific subgroups of veterans who you think are not coming to [this VA facility], e.g., veterans with particular mental health issues (e.g., veterans with MST) or specific demographic characteristics (e.g., women, minorities)? What is it about them or your services or the combination that you think is creating the barrier?
  3. How often do you think these veterans have come for services a couple of times, but then opted not to return? What factors contribute to them dropping out of VA services?
  4. What strategies is [this VA facility] exploring to better engage OIF/OEF/OND veterans who have stopped coming in—or have never come to the VA—for mental health services?

VI. GOING FORWARD

  1. What suggestions do you have for how [this VA facility] can better serve OIF/OEF/OND veterans in the future?
  2. What mental health services does [this VA facility] plan to offer in the future? What is the timeframe for implementation?

VII. CLOSING

Is there anything we didn’t ask you about, but that you think is important for us to know to understand how [this VA facility] serves OIF/OEF/OND veterans?

Thank you for your time.

Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×

IOM VA MH Services Evaluation
OEF/OIF/OND Veterans Who Are VA Service Users

Discussion Protocol
for Individual Interviews and Small-Group Discussions

Thank you for agreeing to participate in this discussion today. My name is ________________ and this is my colleague ________________. We work for Westat, a research organization based in Rockville, MD. Westat is under contract to the Institute of Medicine, part of the National Academy of Sciences, to undertake a Congressionally mandated study assessing the array of mental health services available to OIF/OEF/OND veterans through the VA (for example, individual or group therapy, substance abuse treatment, etc.), and to focus on why some veterans from the wars in Iraq and Afghanistan are not using VA services. Maybe they received mental health services from the VA in the past and stopped, or maybe they have never come to the VA for assistance. Also attending today’s discussion is ________________ [IOM committee member] and ________________ [IOM staff member].

Today we’re interested in hearing your perspectives on these issues. First we want to hear about your experiences accessing services through the VA for any mental health condition: PTSD, depression, substance use disorder, or any other mental health issue you may have faced: How easy or difficult has it been for you to get services and, more importantly, to get services that you think are a good “fit” for you. We’d also like to hear why you think other veterans of the wars in Iraq and Afghanistan are not coming to the VA for mental health support and if there’s anything that could change that: Maybe there’s something the VA could do differently, maybe the veterans need more information about what’s available, or maybe they’re just receiving effective supports elsewhere.

Before we get started there are a few things I should mention. This is a research project. Your participation is voluntary, refusal to participate will involve no penalty or loss of benefits to which you are otherwise entitled, and you may discontinue participation at any time without penalty or loss of benefits to which you are otherwise entitled. If you choose to participate, you don’t have to answer any of our questions that make you uncomfortable. We have planned for this discussion to last no more than 60 minutes.

We will be going to all veterans integrated service networks across the country to see if there are common issues across geographic areas, or if there are strategies or programs being successfully implemented in some locations that could be tried in other venues. After each visit, we will submit a brief, high level report to the VA that summarizes the major findings from the visit. This report will be submitted to the IOM’s public access file for the study. We will also submit reports summarizing our findings across multiple sites to a committee that has been assembled through the Institute of Medicine. [If an IOM committee member is attending the visit, then note that s/he also will prepare a brief report that will be submitted to the IOM’s public access file for the study.] The committee will then incorporate our findings into a larger, overall report that will go to the Department of Veterans Affairs and the Congress. The reports from this study are scheduled to be released to the public in mid 2017. However, your name will NOT be used in any of the reports and no personally identifiable information will be shared with anyone outside of the site visit team here today. I would ask that you all also respect this privacy goal and that whatever is said in this room among you, stays in this room.

To further help us protect your privacy, we have obtained a certificate of confidentiality from the U.S. Department of Health and Human Services. With this certificate we cannot be forced (for example, by court order or subpoena) to disclose information that may identify you in any federal, state, local, civil,

Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×

criminal, legislative, administrative, or other proceedings. The researchers will use the certificate to resist any demands for information that would identify you, except to prevent serious harm to you or others, and as explained below. You should understand that a certificate of confidentiality does not prevent you, or a member of your family, from voluntarily releasing information about yourself, your family, or your involvement in this study. If an insurer or employer learns about your participation, and obtains your consent to receive research information, then we may not use the certificate of confidentiality to withhold this information. This means that you and your family must also actively protect your own privacy. A certificate of confidentiality does not represent an endorsement of the research study by the Department of Health and Human Services or the National Institutes of Health.

The only time we would need to break confidentiality is if we heard that someone was planning to harm him/herself or someone else. The main risk to you in participating is that you may feel uncomfortable sharing your experiences in front of others. We have contact information for the veterans crisis line that we will have available for you at the end of the session.

Do you have any questions? [ANSWER ALL QUESTIONS]

Finally, with your permission, we would like to record this discussion. This recording will be used to help us recall exactly what was said when we go to summarize our findings. The recordings and any notes we have will be stored on Westat’s server. They will be accessible only to the Westat project team. We will destroy the recordings after the study is complete. Are you okay with us recording?

IF PERMISSION WAS GIVEN TO RECORD ASK AGAIN IF THERE ARE ANY QUESTIONS. ANSWER ALL QUESTIONS.

If there are no further questions or concerns, I’d like to start the audio recording now.

TURN ON RECORDER: For the purposes of the recording I am going to ask each of you to state out loud if you are willing to participate in the discussion and if I have your permission to audio tape. GO AROUND THE ROOM AND ALLOW EACH PARTICIPANT TO STATE HIS/HER AGREEMENT TO PARTICIPATE AND BE AUDIO-RECORDED.

I. INTRODUCTIONS

I’d like to start by having each of you introduce yourselves. Please tell us your first name only, what service you were in, when you separated from active duty, and how long you’ve been coming to the VA for mental health services.

II. ACCESS

  1. Let’s first talk about your experiences getting mental health services through [this VA facility]. Think back to when you first contacted the VA about getting support for whatever mental health issue you were facing. What was it like for you getting into that first appointment? PROBE:
    • How easy was it to make an appointment?
    • How long did it take between the time you called and the time you had that first appointment? Was that timing okay with you, or did you think it could have happened more quickly?
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
    • How well did you feel the clinician [e.g., social worker, psychologist] understood your needs at that first appointment?
  1. If you recall, how much time passed between that first appointment and your next scheduled visit? PROBE:
    • Was that a reasonable timeframe or did you feel it should have happened more quickly?
    • Did you see the same therapist or doctor you saw the first time, or someone else? [IF SOMEONE ELSE] Was that okay with you or not? Explain.
  2. Over time, what has the process been like getting follow-up mental health appointments? PROBE:
    • How often have you been able to see your clinician? Does that feel like the right time interval between appointments or would you like it to be longer/shorter? Explain.
    • How easy or difficult has it been trying to make a follow-up appointment?

III. CHOICES

  1. What services or treatment options were offered to you? How much choice do you feel like you were given in the services you received, for example, being able to choose how much time passed between appointments or whether you took medications to address your issue? Explain.
  2. Have any of you ever requested a change in the kind of treatment you’re receiving or asked for a different staff member to help you with your mental health issues? How did staff at this facility respond to your requests?
  3. Have any of you been using any health technologies in your treatment? By health technology we mean things like health apps on your phone, having a video conference call with your therapist rather than coming into the office, or using any online resources.
    • How did you find out about this option? [IF NEEDED, ASK SPECIFICALLY IF CLINICIAN PROVIDED INFORMATION ABOUT IT]
    • What do you like about it? What do you NOT like about it?

IV. QUALITY

  1. How satisfied are you with the quality of the mental health services you’ve received at [this VA facility]? Explain.
  2. Do you think you are getting better as a result of the services you are receiving? Why or why not? How can you tell? [PROBE: Decrease in or less bothersome symptoms? Something you can do now that you were not able to do before you started receiving treatment?]
    IF NOT GETTING BETTER: What do you hope to see change that you haven’t yet?

    What, if anything, do you think you need to help you get better?

  3. What services would you like that are not currently available to you here? Explain.
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×

V. BARRIERS

  1. What made you decide to come to [this VA facility] for mental health services?
  2. Do you have any friends or acquaintances from the Iraq or Afghanistan wars who need mental health services but will not come to the VA for assistance? What reasons have they given you as to why they won’t come here?

    PROBE [IF NOT ALREADY BROUGHT UP]:

    • We have heard that some veterans face significant problems finding a way to get to the VA facility. What, if any, transportation difficulties have you had to overcome to get here? What was your solution?
    • We have also heard that some veterans are worried that they will be stigmatized if people find out they are coming to the VA for mental health services. Are you at all concerned about that? Explain.
  3. What mental health services or supports are any of you receiving outside of the VA? What made you decide to seek those supports outside, rather than inside, the VA?

VI. GOING FORWARD

  1. What suggestions do you have for how [this VA facility] can better serve you and other veterans of Iraq and Afghanistan who are already coming here for support? That is, what will keep you coming back if you need the help?
  2. What could [this VA facility] do to engage those veterans of Iraq and Afghanistan who may still be in need of assistance, but who are not contacting the VA for mental health services?

VII. CLOSING

Is there anything we didn’t ask you about, but that you think is important for us to know to understand your experiences receiving mental health services at [this VA facility]?

Thank you for your time.

Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×

IOM VA MH Services Evaluation
Non-VA Clinical Staff

Discussion Protocol
for Individual Interviews and Small-Group Discussions

Thank you for agreeing to participate in this interview today. My name is ________________ and this is my colleague ________________. We work for Westat, a research organization based in Rockville, MD. Westat is under contract to the Institute of Medicine, part of the National Academy of Sciences, to undertake a Congressionally mandated study of the array of mental health services available to Iraq and Afghanistan war veterans through the VA (for example, individual or group therapy, substance abuse treatment, etc.), and to focus on why some of these veterans are not using the VA services. Maybe they have used the services in the past and stopped, or maybe they have never come to the VA for assistance. Also attending today’s discussion is ________________ [IOM committee member] and ________________ [IOM staff member].

Today we’re interested in hearing your views about why veterans are using the services you provide, and not those offered by the VA. It could be based on your observations or on what you’ve heard from your clients. We would also be interested in any suggestions you may have for how service providers, including the VA, could better engage Iraq and Afghanistan war veterans in mental health services and supports.

Before we get started there are a few things I should mention. This is a research project. Your participation is voluntary, refusal to participate will involve no penalty or loss of benefits to which you are otherwise entitled, and you may discontinue participation at any time without penalty or loss of benefits to which you are otherwise entitled. If you choose to participate, you don’t have to answer any of our questions that make you uncomfortable. We have planned for this discussion to last no more than 60 minutes.

We will be going to all 21 veterans integrated service networks to see if there are common issues across geographic areas, or if there are strategies being successfully implemented in some locations that could be tried in other venues. After each visit, we will submit a brief, high level report to the VA that summarizes the major findings from the visit. This report will be submitted to the IOM’s public access file for the study. We will also submit reports summarizing our findings across multiple sites to a committee that has been assembled through the Institute of Medicine; the committee will then incorporate our findings into a larger, overall report that will go to the Department of Veterans Affairs and the Congress. [If an IOM committee member is attending the visit, then note that s/he also will prepare a brief report that will be submitted to the IOM’s public access file for the study.] The final report from this study is scheduled to be released to the public in mid 2017. Your name will NOT be used in any of the reports and we aim to summarize findings such that comments cannot be attributed to a particular individual. No personally identifiable information will be shared with anyone outside of the site visit team here today. I would ask that you all also respect this privacy goal and that whatever is said in this room among you, stays in this room.

To further help us protect your privacy, we have obtained a certificate of confidentiality from the U.S. Department of Health and Human Services. With this certificate we cannot be forced (for example, by court order or subpoena) to disclose information that may identify you in any federal, state, local, civil, criminal, legislative, administrative, or other proceedings. The researchers will use the certificate to resist any demands for information that would identify you, except to prevent serious harm to you or others, and as explained below. You should understand that a certificate of confidentiality does not prevent

Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×

you, or a member of your family, from voluntarily releasing information about yourself, your family, or your involvement in this study. If an insurer or employer learns about your participation, and obtains your consent to receive research information, then we may not use the certificate of confidentiality to withhold this information. This means that you and your family must also actively protect your own privacy. A certificate of confidentiality does not represent an endorsement of the research study by the Department of Health and Human Services or the National Institutes of Health.

The only time we would need to break confidentiality is if we heard that someone was planning to harm him/herself or someone else.

Do you have any questions? [ANSWER ALL QUESTIONS]

Finally, with your permission, we would like to record this interview. This recording will be used to help us recall exactly what was said when we go to summarize our findings. The recordings and any notes we have will be stored on Westat’s server. They will be accessible only to the Westat project team. We will destroy the recordings after the study is complete. Are you okay with us recording?

IF PERMISSION WAS GIVEN TO RECORD ASK AGAIN IF THERE ARE ANY QUESTIONS. ANSWER ALL QUESTIONS.

If there are no further questions or concerns, I’d like to start the audio recording now.

TURN ON RECORDER: For the purposes of the recording I am going to ask you to state out loud if you are willing to participate in the discussion and if I have your permission to audio tape.

I. INTRODUCTIONS

I’d like to start by hearing a little bit about who you are. Please tell us what your background is (e.g., social work, psychiatry, nursing), what kind of mental health services you provide to OIF/OEF/OND veterans, and how long you’ve been working with the population. Also, please let us know if you are a veteran.

II. ACCESS

  1. How do veterans find out about your services? [PROBE specifically on referrals through the VA]
  2. What percentage of your clients would you estimate to be veterans of the wars in Iraq or Afghanistan?
  3. What is your capacity to serve OIF/OEF/OND veterans? For example, how long is it usually between when they first contact your agency and the time they get in for their first appointment?
    • What options do you have available if your agency is unable to get the veteran in for an appointment in a timely manner?
    • How, if at all, does your capacity to serve veterans impact your ability to serve non-veterans? Explain.
  4. Describe your working relationship with the VA. [PROBE: MOUs? Referrals either direction? IF NO WORKING RELATIONSHIP – Explain.].
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×

III. QUALITY

  1. What services does your agency/organization offer that you believe are most effective at meeting the mental health needs of your OIF/OEF veterans? What evidence do you have that these services are effective? [PROBE: Are the veterans getting better? How do you know? Able to do something after receiving services that could not do when first came in for help?] How do you perceive the quality of services being provided to veterans by other organizations in the community, including the VA? Explain.

IV. CHOICES

  1. Of the many services your agency offers, which do veterans prefer? Why do you think that is?
  2. What options are available to veterans if they are not happy with their clinicians or the services they are receiving?

V. BARRIERS

  1. What kinds of things are preventing OIF/OEF veterans coming in for mental health services? [PROBE ON: stigma; lack of awareness of problems; medical records and job limitations; other issues that have emerged during site visit, e.g., gun ownership]
  2. For veterans who have come for services a couple of times, but then dropped out, what factors contribute to them stopping care?

VI. GOING FORWARD

  1. What suggestions do you have for how your agency can better serve Iraq and Afghanistan war veterans in the future?
  2. What suggestions do you have for how the VA can better engage this population of veterans?

VII. CLOSING

Is there anything we didn’t ask you about, but that you think is important for us to know to understand how your agency serves Iraq and Afghanistan war veterans?

Thank you for your time.

Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×

IOM VA MH Services Evaluation
OEF/OIF/OND Veterans Who Are Not Using VA Mental Health Services

Discussion Protocol
for Individual Interviews and Small-Group Discussions

Thank you for agreeing to participate in this interview today. My name ________________ is and this is my colleague ________________. We work for Westat, a research organization based in Rockville, MD. Westat is under contract to the Institute of Medicine, part of the National Academy of Sciences, to undertake a Congressionally mandated study assessing the array of mental health services available to veterans of the wars in Iraq and Afghanistan through the VA (for example, individual or group therapy, substance abuse treatment, etc.), and to learn why some veterans, like yourself, have either never gone to the VA for these services, or went a couple of times and decided not to go back. Also attending today’s discussion is ________________ [IOM committee member] and ________________ [IOM staff member]. Today we’d like to hear your experiences with or views on the VA and find out if there’s anything you think the VA might be able to do differently to ensure that veterans from the wars in Iraq and Afghanistan have access to needed mental health (mental health and substance use) services and supports.

Before we get started there are a few things I should mention. This is a research project. Your participation is voluntary, refusal to participate will involve no penalty or loss of benefits to which you are otherwise entitled, and you may discontinue participation at any time without penalty or loss of benefits to which you are otherwise entitled. If you choose to participate, you don’t have to answer any of our questions that make you uncomfortable. We have planned for this interview to last no more than 60 minutes.

For this study, we will be going to all veterans integrated service [“health”] networks across the country to see if there are common issues across geographic areas, or if there things being done successfully in some locations that could be tried in others. After each visit, we will submit a brief, high level report to the VA that summarizes the major findings from the visit. This report will be submitted to the IOM’s public access file for the study. We will also submit reports summarizing our findings across multiple sites to a committee that has been assembled through the Institute of Medicine; the committee will then incorporate our findings into a larger, overall report that will go to the Department of Veterans Affairs and the Congress. [If an IOM committee member is attending the visit, then note that s/he also will prepare a brief report that will be submitted to the IOM’s public access file for the study.] The final report from this study is scheduled to be released to the public in mid 2017. Your name will NOT be used in any of the reports and we aim to summarize findings such that comments cannot be attributed to a particular individual. No personally identifiable information will be shared with anyone outside of the site visit team here today. I would ask that you all also respect this privacy goal and that whatever is said in this room among you, stays in this room.

To further help us protect your privacy, we have obtained a certificate of confidentiality from the U.S. Department of Health and Human Services. With this certificate we cannot be forced (for example, by court order or subpoena) to disclose information that may identify you in any federal, state, local, civil, criminal, legislative, administrative, or other proceedings. The researchers will use the certificate to resist any demands for information that would identify you, except to prevent serious harm to you or others, and as explained below. You should understand that a certificate of confidentiality does not prevent

Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×

you, or a member of your family, from voluntarily releasing information about yourself, your family, or your involvement in this study. If an insurer or employer learns about your participation, and obtains your consent to receive research information, then we may not use the certificate of confidentiality to withhold this information. This means that you and your family must also actively protect your own privacy. A certificate of confidentiality does not represent an endorsement of the research study by the Department of Health and Human Services or the National Institutes of Health.

The only time we would need to break confidentiality is if we heard that someone was planning to harm him/herself or someone else.

The main risk to you in participating is that you may feel uncomfortable sharing your experiences in front of others. We have contact information for the veterans crisis line that we will have available for you at the end of the session.

Do you have any questions? [ANSWER ALL QUESTIONS]

Finally, with your permission, we would like to record this interview. This recording will be used to help us recall exactly what was said when we go to summarize our findings. The recordings and any notes we have will be stored on Westat’s server. They will be accessible only to the Westat project team. We will destroy the recordings after the study is complete. Are you okay with us recording?

IF PERMISSION WAS GIVEN TO RECORD ASK AGAIN IF THERE ARE ANY QUESTIONS. ANSWER ALL QUESTIONS.

If there are no further questions or concerns, I’d like to start the audio recording now.

TURN ON RECORDER: For the purposes of the recording I am going to ask you to state if you are still willing to participate in this interview and if I have your permission to audio tape our discussion.

I. BACKGROUND

I’d like to start off by having you tell us a little about your military career. What year you enlisted; what branch of the service were you in; if you were in theater, when and where; and then how long ago you separated from the military.

Briefly, what’s been going on in your life since you separated from the service? (Work? School? Married? Kids?)

II. EXPERIENTIAL NARRATIVE

As I mentioned, we’re interested in learning about the mental health services the VA offers and finding out why some veterans of the wars in Iraq and Afghanistan choose to get mental health assistance from non-VA providers. I’d like you to tell me a story about your experiences getting mental health services that you believe to be a good fit for you. I’m particularly interested in hearing about any experiences you may have had with mental health services through the VA or, if you never contacted the VA, why not. You can start your story wherever you like and you can talk as long as you like, but tell me whatever

Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×

you think is important for me to hear to understand your journey to getting mental health services with which you are satisfied.

POSSIBLE PROBES:

  • Thinking back, what led you to make that first phone call about getting mental health support? (Self-aware? Family or friend said something?)
  • What resources did you use to look for services? (Military One Source? Friend? Internet?)
  • What, if any, experience did you have with the VA?
    • IF NEVER CONTACTED THE VA, ASK WHY NOT. PROBE ON STIGMA, NEGATIVE PERCEPTIONS OF VA SERVICES, ETC.
    • IF WANTED BUT NEVER RECEIVED VA SERVICES
      • Why didn’t you go through the VA for mental health assistance? (Long time to first appointment? Location where there was an available provider too far away?)
      • If [BARRIER DESCRIBED ABOVE] could be taken care of, would you consider the VA if you need mental health services in the future? Explain.
    • IF RECEIVED SERVICES THROUGH VA BUT STOPPED
      • What was your experience with the services you received at the VA?
      • IF POSITIVE EXPERIENCE: Why did you stop?
      • IF BAD EXPERIENCE: What made it a negative experience?
      • If you were to need mental health services in the future, would you consider looking at the VA again? Why or why not?
  • How did you determine that the [SERVICES CURRENTLY RECEIVING] are the best fit for you?

III. GOING FORWARD

What advice would you give to other veterans of the wars in Iraq and Afghanistan who are looking for supports for a mental health disorder?

  • Would you encourage them to contact the VA? Why or why not?

Is there anything else you would like to tell me about your experience? Thank you for your time.

Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×

IOM VA MH Services Evaluation
Family/Friends of OIF/OEF/OND Veterans
Who Are Not Using VA Mental Health Services

Discussion Protocol
for Individual Interviews and Small-Group Discussions

Thank you for agreeing to participate in this interview today. My name is ________________ and this is my colleague ________________. We work for Westat, a research organization based in Rockville, MD. Westat is under contract to the Institute of Medicine, part of the National Academy of Sciences, to undertake a Congressionally mandated study assessing the array of mental health services available to veterans of the wars in Iraq and Afghanistan through the VA (for example, individual or group therapy, substance abuse treatment, etc.). We are also tasked with learning why some of these veterans either never have gone to the VA for these services, or went a couple of times and decided not to go back. Also attending today’s discussion is ________________ [IOM committee member] and ________________ [IOM staff member]. Today we’d like to hear about your experiences trying to encourage a veteran from one of these recent conflicts to seek help for a mental health disorder. We’re interested in where you sought information about services and the veteran’s response to your efforts. We’re also interested in your suggestions for things the VA could do to ensure that veterans from the wars in Iraq and Afghanistan have access to needed mental health services and supports.

Before we get started there are a few things I should mention. This is a research project. Your participation is voluntary, refusal to participate will involve no penalty or loss of benefits to which you are otherwise entitled, and you may discontinue participation at any time without penalty or loss of benefits to which you are otherwise entitled. If you choose to participate, you don’t have to answer any of our questions that make you uncomfortable. We have planned for this interview to last no more than 60 minutes.

For this study, we will be going to all veterans health networks across the country to see if there are common issues across geographic areas, or if there things being done successfully in some locations that could be tried in others. After each visit, we will submit a brief, high level report to the VA that summarizes the major findings from the visit. This report will be submitted to the IOM’s public access file for the study. We will also submit reports summarizing our findings across multiple sites to a committee that has been assembled through the Institute of Medicine; the committee will then incorporate our findings into a larger, overall report that will go to the Department of Veterans Affairs and the Congress. [If an IOM committee member is attending the visit, then note that s/he also will prepare a brief report that will be submitted to the IOM’s public access file for the study.] The final report from this study is scheduled to be released to the public in mid 2017. Your name will NOT be used in any of the reports and we aim to summarize findings such that comments cannot be attributed to a particular individual. No personally identifiable information will be shared with anyone outside of the site visit team here today. I would ask that you all also respect this privacy goal and that whatever is said in this room among you, stays in this room.

To further help us protect your privacy, we have obtained a certificate of confidentiality from the U.S. Department of Health and Human Services. With this certificate we cannot be forced (for example, by court order or subpoena) to disclose information that may identify you in any federal, state, local, civil, criminal, legislative, administrative, or other proceedings. The researchers will use the certificate to resist

Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×

any demands for information that would identify you, except to prevent serious harm to you or others, and as explained below. You should understand that a certificate of confidentiality does not prevent you, or a member of your family, from voluntarily releasing information about yourself, your family, or your involvement in this study. If an insurer or employer learns about your participation, and obtains your consent to receive research information, then we may not use the certificate of confidentiality to withhold this information. This means that you and your family must also actively protect your own privacy. A certificate of confidentiality does not represent an endorsement of the research study by the Department of Health and Human Services or the National Institutes of Health.

The only time we would need to break confidentiality is if we heard that someone was planning to harm him/herself or someone else.

Do you have any questions? [ANSWER ALL QUESTIONS]

Finally, with your permission, we would like to record this interview. This recording will be used to help us recall exactly what was said when we go to summarize our findings. The recordings and any notes we have will be stored on Westat’s server. They will be accessible only to the Westat project team. We will destroy the recordings after the study is complete. Are you okay with us recording?

IF PERMISSION WAS GIVEN TO RECORD ASK AGAIN IF THERE ARE ANY QUESTIONS. ANSWER ALL QUESTIONS.

If there are no further questions or concerns, I’d like to start the audio recording now.

TURN ON RECORDER: For the purposes of the recording I am going to ask you to state if you are still willing to participate in this interview and if I have your permission to audio tape our discussion.

IV. BACKGROUND

I’d like to start off by having you tell us a little bit about yourself: How long have you lived in this area? What kind of work do you do. (For example, do you have any experience with providing mental health services?) And how is the veteran related to you? (Child? Spouse? Friend?)

V. EXPERIENTIAL NARRATIVE

I’d like you to tell me a story about your experiences trying to get your family member/friend into mental health services. I’m particularly interested in hearing about any experiences you may have had with working with the VA or, if you never contacted the VA, why not. I’m also interested in hearing about the veteran’s reactions to any recommendations you tried to make. You can start your story wherever you like and you can talk as long as you like, but tell me whatever you think is important for me to hear to understand what you’ve tried to do for your friend/family member and what you think his/her barriers to service use are.

POSSIBLE PROBES, dependent upon the flow of the narrative:

  • Thinking back, what led you to first talk with your friend/family member about getting help? (Troubling behavior? Worrisome things the person was saying?) How receptive was s/he to that conversation? Explain.
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
  • Did you make any phone calls to try to find services for your friend/family member? IF YES:
    • Whom did you call? How helpful were the people with whom you spoke? Explain.
    • What, if any, experience did you have contacting the VA? IF NEVER CONTACTED THE VA, ASK WHY NOT. PROBE ON STIGMA, NEGATIVE PERCEPTIONS OF VA SERVICES, ETC.
  • What, if any, VA or military resources did you use? How useful were those? Explain.
  • Did your friend/family member make any calls to try to find services? IF YES, PROBE ON WHO WAS CALLED, THE HELPFULNESS OF THE CONTACTS.
  • Did your friend/family member ever contact the VA about receiving services there?
    • IF NO – Why do you think s/he didn’t explore what was available through the VA?
    • IF YES BUT DIDN’T GET SERVICES – What was the reason s/he wasn’t able to obtain services through the VA? [PROBE ON EASE AND TIMELINESS OF GETTING AN APPOINTMENT; ABILITY TO GET TRANSPORTATION TO THE VA; DIDN’T HAVE DESIRED SERVICES]
    • IF RECEIVED SERVICES THROUGH VA BUT STOPPED
      • Why did s/he stop going to the VA? [PROBE ON ABILITY TO GET FOLLOW-UP APPOINTMENT WITH SAME CLINICIAN OR IN A TIMELY MANNER; CONCERN ABOUT STIGMA; DIFFICULTY GETTING TO THE FACILITY]
      • What, if anything, did the VA do to try to re-engage your friend/family member? Why do you think those efforts did not work?
      • What do you think they could have tried to do to re-engage your friend/family member, but did not? Why do you think that might have been successful?

VI. GOING FORWARD

What advice would you give to other family members or friends who are trying to get their friends/family members who are veterans of the wars in Iraq and Afghanistan into services for mental health disorders?

  • Would you encourage them to contact the VA? Why or why not?

Is there anything else you would like to tell me about your experience? Thank you for your time.

Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×

Question Matrix for Ad Hoc Interviews in the Community

Area of Inquiry Questions to Address
Access
  • Are mental health (MH) services at the VA and in the community sufficient to meet the initial and ongoing needs of the veterans of the wars in Iraq and Afghanistan?
  • What MH services are available to veterans, including psychotherapy and pharmacotherapy?
  • At the VA and/or community setting, how long does it take to receive an intake appointment, be assigned a caseworker or therapist, and see a psychiatrist, etc.? How long does it take to get a follow-up appointment?
  • How do VA MH services fit into the local community and local continuum of care?
Pathways to care
  • How do the veterans of the wars in Iraq and Afghanistan find out about, access and utilize mental health services in the VA system? In the community?
  • Do providers offer extended/alternate hours to veterans? If so, are veterans making use of these times or are other accommodations needed?
Engagement
  • What outreach efforts to the veterans of the wars in Iraq and Afghanistan have been successful? Why?
  • Why do veterans discontinue treatment?
  • How inviting and hospitable are VA MH care facilities? How do VA facilities compare to community providers in this respect? What could be done to make facilities more inviting?
Choice
  • What processes are in place at the VA and community clinics for accommodating the values and preferences of the veterans of the wars in Iraq and Afghanistan?
  • What do patients like about the services and what do they dislike? What would increase patient satisfaction with services?
  • What forms of technology are useful to veterans in their mental health services
Barriers
  • What are the challenges that veterans face in getting care for their mental health problems?
  • How might these barriers to treatment be better addressed by the VA? By community providers?
  • How much stigma do veterans who receive mental health treatment face? How do you know?
Quality
  • What is the quality of mental health care provided at the VAMC, small and medium-sized CBOCs, and CBOCs in rural areas?
  • What is the quality of care among community-based mental health service providers?
  • For both VA and non-VA providers, how is quality assessed? What TQI processes are in place specifically to improve mental health services among this population?
  • What is the role of technology in improving the delivery of quality services?
Planning
  • What MH services are VA facilities planning for the future? What services are being planned in the community?
  • How were these services identified?
  • What steps are being taken to increase the engagement of the veterans of the wars in Iraq and Afghanistan in treatment?
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×

NVIVO CODES

ACCESS AND AVAILABILITY
Code Description
Examples of Easy Access Veterans said they had no difficulty getting into the VA for services. Examples include Veterans who transferred from a Warrior Transition Unit (WTU) or Military Treatment Facility (MTF) or who had someone who helped them gain entry.
Navigators Veteran had someone help him or her figure out the VA system, how to enroll, etc.
Navigation Challenges Instances in which the Veteran described not knowing how to get into or through the VA bureaucracy. Also used for discussions that reflect the “total institution” of the military, i.e., that while in the military, a person is told what to wear, when to eat, etc.
Pathways to Care High level category on how Veterans get into mental health care.
Emergency Veteran presents in either a psychiatric or financial crisis
Ultimatum Family member encouraged the Veteran or made an ultimatum (“get help or else I leave”).
Unique Outreach Practices Instances where an interviewee described an outreach strategy that seemed really new and different.
Word of Mouth Friends suggested the individual get help or the person found out about the VA services through word of mouth.
Referrals Out The VA may not have capacity, but does the community and is the VA using it?
Actively Referring For those instances in which VAMCs that had excellent partnerships with community providers, Vet Centers, and the like.
Referral Barriers Things that impeded referring out, even when the VA did not have capacity.
Choice All references to use (or not) of the Choice program should be coded here.
Cost Use this code if the barrier to a referral is the cost of the therapy to the Veteran.
VA Staff Attitudes This code references VA staff who said they would not refer Veterans out of the VA because they didn’t believe the community had the ability to appropriately care for the Veterans. Also code those instances when staff said there were recordkeeping challenges when working with community providers.
Service Capacity High-level code that refers to distinctly negative instances of the system supply being able to meet the demand.
No Capacity Problems Comments from staff or Veterans that indicate they have no difficulty getting people in for appointments.
Things Affected by Category reflects those aspects of service delivery that are impacted by tight capacity.
Appointment Issues Problems with appointments that Veterans and staff noted as a result of the pressures on the system.
Cancellations Those instances in which Veterans discussed cancellations as an apparently structural problem. It was particularly evident when Veterans described having cancelled appointments recoded as “no shows” or finding out the appointment was cancelled when they arrived at the facility.
Frequency Those instances in which follow-up appointments were being scheduled at long intervals (e.g., 3-6 months instead of monthly).
Length Mentions of undesirably short appointments (e.g., 20 minutes with a counselor, 5 minutes with a psychiatrist).
Scheduling For all challenges around scheduling appointments, including admin issues that get in the way, dedicated “clinics” that can’t be changed in the scheduling system, etc.
Type of Appointment Veteran is unable to get the type of appointment s/he wants. This may be group instead of individual counseling, or psychiatry and no counseling.
Type of Therapist Use specifically for male vs. female or psychiatrist vs. psychologist (or psychologist vs. social worker).
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Code Description
Lack of Space Not enough, wrong type, no privacy, dangerous
Wait Times Instances in which interviewee said Veterans have to wait a long time (subjective or objective comment) to get into services.
Things Affecting Codes here refer to a small set of issues that were perceived to be contributing to a backlog in the system.
Admin Responsibilities Mentions of clinical staff’s administrative responsibilities either taking up clinical time, too much time, or having to be done in off hours.
Caseload References to the difficulties of large caseloads.
Blockers References to people “clogging up” the system, thus preventing appointments from being available to others. Usually this is due to people who request long-term therapy, but could be due to other things.
HR References to HR hiring practices, particularly the slowness with which they fill open positions.
No Shows Staff in several sites complained that more services would be available if patients actually showed up for their appointments. Also code those instances in which Veterans indicated their appointments were coded as a “no show” inappropriately.
Staff Turnover This code should be used when tagging comments about high staff turnover, e.g., regular employees OR interns coming in and leaving after a short period of time (sometimes “to get their card stamped” for other VA positions).
BARRIERS
Code Description
Bad Reputation of VA Various instances in which interviewees reported the VA has a negative public image.
Media Bad reputation via media reports, press coverage, negative incidents.
Rumor Mill Use for instances of the rumor mill among peers or the local community (not relatives).
Sins of the Father Specific cases when Veterans mentioned that they did not want to go to the VA because a father, brother, uncle, grandfather had had a bad experience.
Barriers NOS “Barriers not otherwise specified”—for barriers that do not fit into the extant categories.
Being Around Other People Veterans report that they do not want to go to the VA because they cannot be around others.
Built Environment This denotes a broad array of things including inadequate parking, a structure or even internal set-up that is a trigger for someone with PTSD, or overcrowding/uncomfortable space. Issues around the facility having a reputation as a gathering place for homeless people should be coded under the Welfare code, below.
Childcare Veteran cannot get to services because either cannot arrange or pay for childcare.
Civilians Comments from Veterans about civilians (for positive comments, double-code)
Communication This code should be used for all instances in which interviewees describe either communication (phone, email, text) challenges or successes. Telehealth can also be coded to this category.
Competing Demands This refers to work, school, family demands only with the result being the Veteran does not have time to take care of him/herself.
Customer Service Code both positive and negative customer service experiences here. In general this has come up in reference to admin staff (MAS), not clinicians.
Disability Compensation Used for those comments suggesting Veterans are reluctant to seek care because it may reduce their disability rating and compensation.
DoD to VA References to the linkage between getting out of active duty and becoming a Veteran. May be about records, DD214, communication, etc.
Eligibility Determination of eligibility—and issues around that (e.g., timing, lost in the system, poor communication)—as a barrier to care.
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Code Description
Employment This code refers specifically to concerns Veterans expressed about how receipt of MH care would affect their jobs or possibilities for employment, including their position in the National Guard or their reenlistment possibilities. It should include things like not being able to obtain security clearances or driver’s licenses as well.
Financial Distress Instances in which financial distress of Veterans was discussed in relation to care.
Guns Veteran will not seek care because of concern about loss of weapons or inability to purchase weapons.
Actual Loss of Weapons Instances in which either the interviewee or someone s/he knew directly lost his/her firearms as a result of seeking mental health care. Rumors are not included under this code, only those instances that are based on first-hand knowledge.
Information Privacy Respondent expressed concern that health records or other information will not be kept private by the VA.
Knowledge Gap Refers specifically to information obtained (or not) at the point of transition from DoD. Also include instances in which the individual did not know s/he was a Veteran or believed that the VA was for someone else (e.g., older people, people who were injured in combat). It does not refer to navigational challenges per se.
Bad Information Veteran was misinformed about care, eligibility for care (double code), where to go, etc.
LGBT Specific Issues Specific discussions related to LGBT access to care.
Military Cultural Expectations Refers to carryover from the Veteran’s experience in the military—“suck it up, buttercup,” worries that the docs will be as bad as military docs, worries that there will be no confidentiality like in DoD, etc.
Not Aware of Need Veteran indicates not seeking treatment because of a lack of perceived need. “I’m fine.”
Service Hours For all cases in which the speaker talks about the hours during which VA services are being offered AND Vet Center or community-based services.
Stigma About MH Care Refers both to internal (“I thought I was a failure”) and external (“I was embarrassed”) stigma.
Transportation Any reference to transportation challenges, including lack of a car, gas money, public transportation, inability to drive because of PTSD, etc. Include any references to travel reimbursement here.
VBA to VHA Issues Confusion between VBA and VHA.
Welfare Carryover Instances in which speaker’s comment suggests a perception - either on their part or about them by someone else - that the VA and its services constitute a welfare system with cheaters in it. Comments about the front waiting area looking like everyone is homeless should be added to this category.
Women-Specific Issues Issues raised that are keeping women from seeking services at the VA.
Positive Additional code if something that is labeled as a barrier, but interviewee indicates it has been overcome in a particular instance. For example, “We have excellent childcare services here” should get double-coded “childcare” and “positive.”
PROMISING PRACTICES (PP)
Code Description
PP Call It Something Else Examples where creative terminology has been used instead of a standard term that has a negative connotation for Veterans (“psycho-education class” vs. “treatment orientation group”).
PP Customer Service Examples of positive customer service.
PP Peers Examples of peer support services that are perceived to be effective.
PP Positive Ads Examples of positive marketing.
PP Preventing Dropout Examples of effective strategies to reduce premature drop-out.
PP Process Review Examples where the VA has undertaken a review of procedures and made adjustments on the basis of findings.
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Code Description
PP Scheduling Examples of scheduling system that works.
PP Spouse Outreach Examples of reaching out to spouses (education) to encourage them to bring Veteran in for care.
PP Telehealth Examples of effective telehealth.
PP Women’s Issues Examples where services have been adjusted to meet women’s needs.
QUALITY
Code Description
Drop Out Veteran quits treatment for one reason or another.
Modification to Evidence Based Therapy (EBT) Code all instances in which VA staff or Vet Center staff indicate that they have somehow modified the EBT they are discussing.
Efficiency Refers to a dimension of quality that ensures that resources are not wasted unnecessarily. Examples are of efforts on the part of the VA to be “good stewards of the resources,” for example, groups versus individual therapy, use of time-limited EBTs instead of individual therapy without time limits, etc.
Getting Better Instances in which Veterans said the quality was good because they have improved, or in response to what “getting better” would look like.
Satisfaction Broad category to reflect Veterans’ discussions about their satisfaction or dissatisfaction with services received.
Not Satisfied Specific issues identified by Veterans as resulting in unsatisfactory clinical experiences. ACCESS or APPOINTMENT CHALLENGES not coded here.
Bad Fit Interviewee indicates that there was a “bad fit” between the Veteran and the clinician.
Lack Cultural Competence Instances in which Veterans said their clinicians did not have an adequate understanding of military culture.
Medication Management Interviewee expressions of dissatisfaction with medication management.
Negative Clinical Demeanor All references to ACTIONS on the part of the clinician that were unsatisfactory to the Veteran. These include not making eye contact during the appointment, not listening to the Veteran (e.g., not paying attention or not giving the Veteran time to speak), being dismissive (“I’ve had other patients worse than you”), etc.
Overwhelming Statements about how the therapies offered by the VA are overwhelming to Veterans without preparation, or without proper social support.
NEG Young Clinician Specific references to the clinician being too young (from the Veteran’s perspective) and thus not mature enough or experienced enough to deal with Veterans’ issues.
Satisfied
Camaraderie Use this code when a Veteran or clinician mentions that camaraderie in treatment (usually in groups) led to good treatment.
Cultural Competence Instances in which clinician was said to understand the military or when Veterans discuss peer support groups.
Goes the Extra Mile Veterans said they felt like they mattered to the clinician (as a unique person). This includes returning phone calls, remembering anniversary dates, adjusting appointment times, etc.
Patient-Centered Examples from Veterans and clinicians that describe satisfaction with services because they are patient-centered—that is—they meet the preferences of the Veteran.
Positive Clinical Demeanor Use for the positive examples of eye contact, listening, hearing, etc., but also for those cases in which the Veteran said, “She cares about me,” He believes in me,” “I trust him.”
POS Young Clinician Veteran discusses feeling positively about a young, new clinician because he or she is fresh and enthusiastic.
Wellness Activities Instances in which the VA or a community provider was using a range of therapies, including yoga, meditation, etc. for healing.
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
RECOMMENDATIONS
Code Description
Better Data Suggestion that better data would allow for more appropriate treatment.
Customer Service Hire More Staff Suggestions around improving the customer service experience at the VA.
Case Managers Interviewee said the VA needs more case managers.
Clinical Staff Interviewee said the VA needs more clinicians (psychologists, prescribers, RNs).
Support Staff Interviewee suggested the VA hire more admin/support staff.
Improve Communication
Alternate Communication Recommendations for alternative ways for the VA and Veterans to communicate. This includes texting, better phone systems, people answering the phones, etc.
VA Internal Communication Recommendation for improved communication within the VA (either within one facility/system or between systems).
Marketing Word of Mouth Recommendation that the VA needs to do a better job advertising its services, eligibility, etc. to Veterans.
Gun Information Specific recommendations that the VA indicate the rules about seeking mental health treatment and the impact on gun ownership.
Out-Processing Changes Recommendations for how out-processing from active duty to the VA could be improved for the Veteran.
DOD Link to VA Specific discussions about the need to better link the DOD and the VA, including record transmission.
Private Sector Broad code regarding recommendations for the VA’s connection with private sector care. Included under this heading are comments about outsourcing Veterans, use of the Choice program, and the need for the VA to work more collegially with private sector organizations.
Recommendations for Therapies Recommendations specifically having to do with treatment, therapeutic options, etc.
Alternative Therapies Recommendations for non-traditional treatments, including yoga, mindfulness, use of medical marijuana, etc.
Promote Recovery Comments about the VA’s need to take a more recovery-oriented approach to care (or strengths-based orientation).
Recommendations for MST Recommendations about better treatment for survivors of Military Sexual Trauma (MST).
Spouse or Family Suggestions that the VA needs to begin to treat the whole family, not just the Veteran.
Recommendations for Women Specific recommendations to improve access to care for women Veterans.
Reduced Bureaucracy Recommendations for getting around the bureaucracy, which is seen as a barrier to efficient and effective treatment, scheduling, etc.
Accountability Sense that the VA does not get rid of staff who perform poorly on the job. In general, Veterans mentioned this with respect to clerks and front-line customer service, although it was also raised with respect to clinicians.
Flexible Hours Suggestions that the VA have greater flexibility in when it offers services. In some locations, VA staff noted that their building contracts prevent the facility from being open outside the contracted hours.
Local Flexibility Recommendations for more local control of the service delivery system (including treatments, hours, etc.).
Scheduling Recommendations for how to improve the way appointments are scheduled.
Staff Appreciation Recommendations for how to improve staff morale.
Take Services to Patients Suggestions that the VA take more services out to the patients rather than requiring patients to come into a central facility for care. There was some discussion of more mobile vans, but other “out service” suggestions are also included here.
Training Recommendations for where training (generally) could be improved.
VBA Recommendations Specific recommendations for making sure VBA and VHA are clearly separated for the Veterans.
Veteran Employees Volunteers Recommendations for increased presence of Veterans at the VA, either as regular employees, peer supports, greeters, patient advocates, and the like.
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×

This page intentionally left blank.

Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 417
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 418
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 419
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 420
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 421
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 422
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 423
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 424
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 425
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 426
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 427
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 428
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 429
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 430
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 431
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 432
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 433
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 434
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 435
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 436
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 437
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 438
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 439
Suggested Citation:"Appendix B: Supporting Documentation for the Site Visits: Questionnaires and NVivo Codes." National Academies of Sciences, Engineering, and Medicine. 2018. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press. doi: 10.17226/24915.
×
Page 440
Evaluation of the Department of Veterans Affairs Mental Health Services Get This Book
×
Buy Paperback | $95.00 Buy Ebook | $74.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Approximately 4 million U.S. service members took part in the wars in Afghanistan and Iraq. Shortly after troops started returning from their deployments, some active-duty service members and veterans began experiencing mental health problems. Given the stressors associated with war, it is not surprising that some service members developed such mental health conditions as posttraumatic stress disorder, depression, and substance use disorder. Subsequent epidemiologic studies conducted on military and veteran populations that served in the operations in Afghanistan and Iraq provided scientific evidence that those who fought were in fact being diagnosed with mental illnesses and experiencing mental health–related outcomes—in particular, suicide—at a higher rate than the general population.

This report provides a comprehensive assessment of the quality, capacity, and access to mental health care services for veterans who served in the Armed Forces in Operation Enduring Freedom/Operation Iraqi Freedom/Operation New Dawn. It includes an analysis of not only the quality and capacity of mental health care services within the Department of Veterans Affairs, but also barriers faced by patients in utilizing those services.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!