Q1 2023

What we learned from this quarter's data:

EDU’s review of 1st quarter data identified:

• Ease/Simplicity and Efficiency/Speed continue to score at the highest of all domains; however, Confidence/Trust has grown to one of the highest domains as well.
• The majority of respondents indicated the tools VA makes available to them to make education and training decisions was not applicable in their decision to apply for benefits. This is a pain point because EDU’s Confidence/Trust score depends on the response to this question. However, it’s not apparent whether or not respondents were aware of the tools, or the tools simply were not useful.
• Equity/Transparency is the lowest score domain in each survey, except for Touchpoint 3 (“Receiving Benefits”) in which Satisfaction is the lowest score.

Moments That Matter to Veterans:

• Satisfaction improved for each survey. Confidence/Trust scores also improved, but not as significantly across all surveys.

Future initiatives:

• EDU continues to re-engineer the current VSignals surveys, in which we expect response rates will increase.
• EDU expects Equity/Transparency scores to increase as VBA implements the Digital GI Bill Initiatives.

OAR’s review of 1st quarter data identified:

• Customer/Veteran satisfaction remains higher for Higher-Level Review (HLR) claimants participating in an informal conference. Fifty-one percent of HLR survey respondents reported participating in an informal conference. Sentiment and average scores are similar at the two Decision Review Operation Centers. •The first survey question (“Please indicate how you filed your decision review request”) does not have an option to report the claim was filed online. The only options are by mail or fax, dropped it off at a VA Regional Office, or my private attorney or Veteran Service Officer filed it for me. Claimants that filed online feel excluded or overlooked.

Two Insights from Moments That Matter to Veterans:

• Satisfaction with the HLR process is influenced by the decision review outcome. Comparison of cases where all issues were granted to cases where all issues were denied shows a significant positive sentiment trend for those cases where all issues were granted. This impacts satisfaction significantly as the grant rate is and will remain low. The dashboard only included decision review results for 8.8% of the Q1 cases. This data is populated from a CaseFlow-generated table within Enterprise Data Warehouse that historically has failed to provide adequate data. Claimants want to be part of the claim process and prefer to have the status of their claim communicated throughout the process. Filing a claim, waiting/not understanding what or if something is happening, and then receiving the decision by mail is perceived by the claimant as impersonal and exclusionary.
• Claimants would like an advocate within VBA to aid with lane choice (i.e., someone who can recommend the best lane and strategy for their individual situation). Service Recovery (SR)/Process Improvement (PI) respondents report the necessity for third-party assistance to choose a lane and file the appropriate claim. Most SR/PI respondents report they need assistance with navigating the process, as the filing process is not intuitive for all claimants.

Future initiatives:

• OAR will continue reviewing the trend analysis on Veteran outcomes and seek ways to improve satisfaction scores.

Service details

Q4 2022

What we learned from this quarter's data:

EDU’s review of 4th quarter data identified:

• EDU saw a decrease in response rates in Q4 due to an interruption in survey distribution during the month of August.
• Ease/Simplicity and Efficiency/Speed continue to score at the highest of all domains.
• ECC time to connect to a phone representative continues to be a pain point, as indicated by satisfaction score decrease in Q2, Q3, and Q4.
• Quality continues to be the lowest scoring domain.

Moments That Matter to Veterans:

• “Applying for Benefits” and ECC trust scores both decreased from Q3 to Q4
• “Receiving Benefits” survey responses continue to increase from Q3 to Q4

Future initiatives:

• EDU continues to re-engineer the current VSignals surveys, in which we expect response rates will increase.
• VBA continues to address staffing shortages related to the ECC and has recently shifted ECC operations to OFO, which may reduce the time-to-connect pain point.
• EDU expects Quality scores to increase as VBA implements the Digital GI Bill Initiatives.

OAR’s review of 4th quarter data identified:

• Feedback from VSignals AMA FY22 Q4 surveys show customer satisfaction is higher with Veterans who are participating in informal conferences.
• Scheduling informal conferences remains a pain point for HLR participants; 48% of HLR survey respondents participated in an informal conference (IC).

Two Insights from Moments That Matter to Veterans:

• Customer/Veteran satisfaction scores continue to be related to the decision of the Higher-Level Review. A comparison of cases where all issues were granted versus cases where all issues were denied shows a significant positive sentiment trend for those where all issues were granted.
• Scheduling informal conferences remains a pain point for HLR participants based on inconsistent communication. There have been documented instances where Veterans have felt pressure to engage in an IC on the spot, miss a call from VA in an attempt to schedule an IC, or even talk to someone from a DROC who informs the Veteran that they are only collecting a statement and another employee will hold the IC (but no other contact with the Veteran is subsequently made).

Going Forward:

• The HLR decision process trends do not support the expectation that satisfaction scores will increase over time, as the data does not project the grant rate to increase significantly over time and the quality of AOJ decisions is expected to increase over time.
• A new scheduling system is in works for the IC pain point. In time, it is expected that the new scheduling system will mitigate this hurdle.
• OAR will continue focusing on trends that are influenced by Veteran expectations tied to actual outcomes, with the intent to impact satisfaction scores.

Service details

Q3 2022

What we learned from this quarter's data:

OAR’s review of 3rd quarter data identified:

• Customer satisfaction remains higher for Veterans/customers participating in an informal conference. • Free text comments reveal some frustration related to the communication and scheduling of these informal conferences. • Satisfaction with the HLR process is influenced by decision review outcome (i.e. if decision is granted, satisfaction score increases; if decision is denied, satisfaction score decreases).

Two insights from Moments That Matter to Veterans include:

• VSignals AMA FY22 Q3 surveys revealed a strong trend of Veterans/customers providing compliments related to how we explained the higher-level review (HLR) process during informal conferences. • Comparison of cases where all issues were granted compared to cases where all issues were denied shows a significant positive sentiment trend for those cases where all issues were granted. This has an impact on Veteran satisfaction with the HLR process and impacts our scores.

Going forward:

• The nature of the HLR decision process does not lead us to believe that satisfaction scores will increase over time because we do not expect the grant rate to increase significantly over time. • OAR will continue reviewing the trend analysis on Veteran outcomes and seek ways to improve satisfaction scores.

EDU’s review of 3rd quarter data identified:

• Ease/Simplicity and Efficiency/Speed have consistently scored the highest of all domains. • The overall EDU survey response rates increased noticeably in Q3. • ECC time to connect to a phone representative emerged as a new pain point as indicated by satisfaction score decrease in Q2 and again in Q3. • Quality continues to be the lowest score domain.

Two insights from Moments That Matter to Veterans include:

• “Receiving Benefits” substantially increased survey responses from Q2 to Q3. • “Apply for Benefits” and ECC trust scores both decreased for Q2 to Q3.

Going forward:

• EDU expects to see gains in the Quality Domain as VBA continues to deploy technical and automation improvements as part of the Digital GI Bill initiative. • VBA is addressing staffing shortages related to the ECC time-to-connect pain point. • EDU expects trust scores to increase following an annual pattern as students return to school in Q4.

Service details

Q2 2022

What we learned from this quarter's data:

A review of the comments from Q2 showed keywords such as “concern” and “frustration” were occurring at the same or similar frequency as “helpful” and “clear.” This suggests there is an inconsistent distribution of educational materials or absorption of Appeals Management Act (AMA) related to the Higher Level Review (HLR) purpose and process. However, it should be noted that with this suggestion there were multiple comments expressing gratitude with the VA decision maker explaining everything to eliminate issues and hurdles.

There are some concerns, based on comments, that informal conferencing scheduling is inconsistent. Veterans have voiced “playing phone tag” with the VA which suggests a more aggressive and standardized approach to scheduling hearings should be considered. A review of the CX feedback from FY22 Q1 found a strong trend in the word “helpful” and “clear” from multiple Veterans that have used the HLR process. This is supported by CX feedback from FY22 Q1 that found a strong trend in complimenting how issues and events in the HLR process were explained which enhanced the Veteran experience. Professional behaviors with VA personnel operating in the HLR process showed consistency with Q2 as it did with Q1 with comments both complimentary and concerning operating within the same feedback range as it did in Q1. Overall this suggests VA personnel are doing the correct actions at the beginning, with room to grow in operational aspects. Informal Conferences continue to be identified as a hurdle with regards to scheduling. Comments have been identified as frustration within the communication of these informal conferences. However, it cannot be concluded if this is a point of view difference or an actual issue that should be addressed. It can be suggested, however, that a more aggressive approach to scheduling can be recommended. No new pain points have been identified at this time.

This quarter, the Enrolling in School survey saw substantial increases over the other two benefits surveys. Additionally, as did the Applying for Benefits survey question about the usefulness of the GI Bill Comparison Tool saw an increase; however, the increase was expected based on GI Bill Comparison Tool updates made this year. Satisfaction scores dropped while trust scores remained the same. However, satisfaction scores have increased and are in line with trust scores again. Quality continues to be the lowest-scoring domain, though it increased this quarter. We believe Quality is being affected by the perception of uncertainty caused by system upgrades and the deployment of the enrollment verification requirement. As students become acclimated to those changes, their perceptions of Quality seem to be improving, especially as we identify and address system issues caused by modernization. Additionally, the process after submitting an application or receiving a Certificate of Eligibility also continues to score lower. Call wait times became a new pain point this quarter due to enrollment verification, though that score was increasing by the end of the quarter. VBA is reviewing all aspects of its Education VSignals collection, analysis, and decision-making processes this year. VBA is also continuing to deploy technical and automation improvements as part of the Digital GI Bill initiative that should improve scores in the Quality domain.

Service details

Q1 2022

What we learned from this quarter's data:

   This quarter VBA’s scores increased in the Trust and Effectiveness domains; Employee Helpfulness and Ease/Simplicity are the highest-scoring domains. Through faster internal communications, facilitated by technology, VBA was able to expedite claims that were pending additional information or clarification.   VBA is enhancing the Ease/Simplicity domain by employing the GI Bill Journey communications campaign and implementing new technical capabilities to improve personalized information access. VBA is also updating the Call Center platform to resolve technology issues concerning ease of access. 
    Examples of pain points include a preference that informal conferences be scheduled rather than conducted at the time of first contact; terminology being used by the specialist; and estimated completion time provided to callers. VBA is addressing these pain points through service recovery feedback and refresher trainings, which include listening to call recordings.

Service details

Q4 2021

What we learned from this quarter's data:

Service details

Q3 2021

What we learned from this quarter's data:

Service details

Q2 2021

What we learned from this quarter's data:

Data Highlights: NCC - Efficiency increased by 10 percentage points over the past year; All domains are steadily increasing each quarter ECC - Employee helpfulness remains over 90%
INS - All domains increased by 3 percentage points or more; Satisfaction increased by 9 percentage points EDU - Equity had a 22 percentage point correction from FY21 Q1 with applying for benefits OAR - Operational data still under review and validation

Service details

Q1 2021

What we learned from this quarter's data:

VBA National Call Center (NCC) data highlights: 89% of Veterans find the NCC employees helpful Efficiency increased from 60% to 77% from FY20 Q1 to FY21 Q1

Operational data still under review and validation for VBA’s EDU Call Center and EDU Service surveys. This entry will be updated when data is cleared.

Service details