Q3 2023

What we learned from this quarter's data:

As reported previously, OPM’s survey provider migrated from a legacy platform to a FedRAMP platform in Q2 FY 2023, which led to sizable gaps in data for the desktop and mobile surveys. While OPM collected complete results for both surveys in Q3 FY 2023, it is still possible that there are lingering effects on this quarter’s results related to the migration due to initial differences in survey questions and scoring protocol which were subsequently reverted to what was previously shown on the legacy platform.

In Q3 FY 2023, the overall satisfaction scores for both the USAJOBS desktop and mobile applications decreased compared to Q1 FY 2023 results. The overall satisfaction score for USAJOBS Desktop decreased from 3.95 to 3.91, while the overall satisfaction score for USAJOBS Mobile decreased from 4.27 to 4.14 from Q1 FY 2023 to Q3 FY 2023.

The scores for the drivers of satisfaction (effectiveness, ease, efficiency, transparency, and website helpfulness) in both the desktop and mobile surveys also decreased almost across the board from Q1 FY 2023 to Q3 FY 2023, the one exception being a 0.05 point increase in desktop transparency. On average, desktop scores fell .03 points, while the average decrease in mobile drivers of satisfaction scores dropped by .13 points. Further study is required to determine whether these decreases are lingering effects of the survey platform migration, as there are no obvious drivers of such change such as major system issues.

While overall satisfaction with the USAJOBS help desk fell 0.13 points from Q2 to Q3 FY 2023, most of the drivers of satisfaction scores increased significantly: trust (2.86 to 3.30), effectiveness (2.67 to 3.39), ease (2.71 to 3.34), efficiency (2.84 to 3.16), and employee helpfulness (2.96 to 3.13). On the other hand, the score for transparency fell 0.54 points (3.27 to 2.73).

In one year of data collection, scores for the help desk survey, which has a small sample size, have been quite volatile, with significant variations in either direction from quarter to quarter.

Service details

Q1 2023

What we learned from this quarter's data:

In Q1 FY 2023, the overall satisfaction scores for both the USAJOBS desktop and mobile applications increased slightly compared to Q4 FY 2022 results. Continuing a trend of slight increases beginning in Q2 FY 2022, the overall satisfaction score for USAJOBS Desktop increased from 3.89 to 3.95, while the overall satisfaction score for USAJOBS Mobile rose from 4.25 to 4.27 from Q4 FY 2022 to Q1 FY 2023.

The scores for the drivers of satisfaction (effectiveness, ease, efficiency, transparency, and website helpfulness) in both the desktop and mobile surveys also increased slightly from Q4 FY 2022 to Q1 FY 2023. Individual scores for these questions, as well as the confidence/trust question, increased by an average of 0.05 for the desktop survey and 0.04 for the mobile survey. While further research is needed, these positive trends may be the result of changes made by OPM to the site in responses to data collected from customer experiences.

In Q1 FY 2023, six of the seven scores collected in the USAJOBS Help Desk Feedback survey increased compared to Q4 FY 2022, and the overall satisfaction score rose by 0.17 points (2.85 to 3.02). The one exception was transparency, which decreased slightly from 3.35 to 3.31, though this was still the highest scoring question on the survey. Continuing a trend set in past quarters, individual responses for each question tended to be very positive or very negative, with fewer middle-range responses. This division in responses is likely caused by respondents contacting the help desk when they encounter a problem and then responding to the survey only after the help desk either resolves or cannot resolve their issue. As this is the second quarter that data has been collected for this survey, OPM requires additional data to determine whether this quarter’s increases represent a broader trend of improvement. To improve customer satisfaction, the USAJOBS help desk staff is collecting responses in real time and contacting submitters who were not satisfied to drive improvements to satisfaction.

Service details

Q4 2022

What we learned from this quarter's data:

In Q4 of FY 2022, the overall satisfaction scores for both the USAJOBS desktop and mobile applications stayed consistent with the scores in Q3 of FY 2022 as well as those of FY 2021. For the desktop survey, the overall satisfaction score rose from 3.88 to 3.89 from Q3 to Q4. For the mobile application, the score rose from 4.23 to 4.25 from Q3 to Q4. Changes to scores were minimal.

The scores for the drivers of satisfaction (effectiveness, ease, efficiency, transparency, and website helpfulness) in both the desktop and mobile surveys also stayed within a consistent range from Q3 to Q4 of FY 2022. Individual scores for these questions, as well as the confidence/trust question, mostly showed no change or small increases from FY 2022 Q3 to Q4. All score changes were minimal (less than 0.05).

As the USAJOBS Help Desk Feedback survey was deployed for the first time this quarter, there is no past performance for which to compare scores. In Q4, average scores varied from 2.69 (effectiveness) to 3.35 (transparency). Generally, individual responses for each question tended to be very positive or very negative, with fewer middle-range responses. This may be caused by the fact that respondents tend to contact the help desk when they encounter a problem and will then respond to the survey after the help desk either resolves or cannot resolve their issue. The USAJOBS help desk staff have been collecting responses in real time and contacting submitters who were not satisfied, which may positively impact scores in the future.

Service details

Q3 2022

What we learned from this quarter's data:

In Q3 of FY 2022, the overall satisfaction scores for both the USAJOBS desktop and mobile applications stayed fairly consistent with the scores in Q2 of FY 2022 as well as those of FY 2021. For the desktop survey, overall satisfaction scores rose from 3.85 to 3.88 from Q2 to Q3. For the mobile application, the score for both Q2 and Q3 was 4.23. Changes to scores were minimal or nonexistent.

The scores for the drivers of satisfaction (effectiveness, ease, efficiency, transparency, and website helpfulness) in both the desktop and mobile surveys also stayed within a consistent range from Q2 to Q3 of FY 2022. Individual scores for these questions, as well as the confidence/trust question, mostly showed no change or small increases from FY 2022 Q2 to Q3. All score changes were minimal (less than 0.05).

Service details

Q2 2022

What we learned from this quarter's data:

Overall satisfaction: In Q2 of FY 2022 the satisfaction scores for both the USAJOBS desktop and mobile applications stayed fairly consistent with that of Q1 of FY 2022 as well as those of FY 2021. For the desktop survey, overall satisfaction scores dropped from 3.9 to 3.85 from Q1 to Q2. For the mobile, however, it rose from 4.17 (Q1) to 4.23 (Q2). In both cases, the changes of .05 points were minimal.

The drivers of satisfaction (effectiveness, ease, efficiency, transparency, and website helpfulness) in both the desktop and mobile surveys also stayed within a consistent range from Q1 to Q2 of FY 2022. In all cases for these questions, as well as the Confidence/Trust question, scores rose slightly for the mobile survey and dropped slightly for the desktop survey from FY 2022 Q1 to Q2. All score changes were minimal.

Service details

Q1 2022

What we learned from this quarter's data:

Overall satisfaction: In Q1 of FY 2022 the satisfaction scores for both the USAJOBS desktop and mobile applications stayed fairly consistent with all quarters of FY 2021. For the desktop application, overall satisfaction scores were at 3.9, which was higher than any quarter of FY 2021. In FY 2021, desktop satisfaction ranged from 3.82 (Q2) to 3.89 (Q3). However, for the mobile application, overall satisfaction scores were at 4.17, which was lower than any quarter of FY 2021. In FY 2021, mobile satisfaction ranged from 4.18 (Q3) to 4.23 (Q1).

Desktop: The drivers of satisfaction (effectiveness, ease, efficiency, transparency, and website helpfulness) were fairly consistent from FY 2021 Q4 to FY 2022 Q1, with all questions exhibiting a slight positive trend. The efficiency question (it took a reasonable amount of time to do what I needed to do) showed the greatest increase in this timeframe (3.73 to 3.86).

Mobile: The drivers of satisfaction (effectiveness, ease, efficiency, transparency, and website helpfulness) were also fairly consistent quarter to quarter, with the responses to most questions exhibiting a slight negative trend from FY 2021 Q4 to FY 2022 Q1. The greatest decreases were in response to the effectiveness question (my need was addressed) (4.11 to 4.07) and the efficiency question (it took a reasonable amount of time to do what I needed to do) (4.07 to 4.03).

Service details

Q4 2021

What we learned from this quarter's data:

Overall satisfaction: In FY 2021, for both the USAJOBS desktop and mobile applications, scores stayed fairly consistent. For the desktop application, overall satisfaction ranged from 3.82 to 3.89, while for the mobile application, satisfaction ranged from 4.18 to 4.23.

Desktop: The drivers of satisfaction (effectiveness, ease, efficiency, transparency, and website helpfulness), were fairly consistent quarter to quarter, with the responses to most questions exhibiting a slight positive trend. The effectiveness question (my need was addressed) showed the greatest increase from Q1 to Q4 (3.61 to 3.73) followed by the transparency question (I was easily able to access the website on my devise) (4.06 to 4.17).

Mobile: The drivers of satisfaction (effectiveness, ease, efficiency, transparency, and website helpfulness) were also fairly consistent quarter to quarter, with the responses to most questions exhibiting a slight negative trend in Q2 and Q3, but increasing again in Q4. Responses to the efficiency question (it took a reasonable time to do what I needed to do) showed the greatest change in scores between the four quarters (4.21, 4.03, 4.03, and 4.07 respectively).

Service details

Q3 2021

What we learned from this quarter's data:

The desktop survey scores generally increased across all CX measures from Q1 to Q3 in 2021. The two exceptions were a .05 point drop for the satisfaction question, and no change for the trust/confidence question from Q1 to Q2. The satisfaction and trust/confidence scores were highest in Q3 when compared to both Q1 and Q2.

USAJOBS released the job status feature in Q2, and its initial release may have contributed to the Q2 drop in the satisfaction question score. The score leveled off in Q3 which is a potentially good indicator that users are becoming more comfortable with the new feature.

In Q2, USAJOBS also updated the desktop survey code to fix errors that occurred when the survey was presented to users. This led to the survey response count almost doubling between Q1 and Q2 and more than doubling from Q2 to Q3.

The mobile survey scores stayed consistent for the ease and efficiency questions, and had slight drops in the scores for all other questions from Q2 to Q3. These changes were relatively small, with the effectiveness question having the largest drop at .05 points. All scores have remained above 4 points, with the mobile survey generally scoring higher than the desktop survey across all CX measures.

Service details

Q2 2021

What we learned from this quarter's data:

USAJOBS Desktop (Summary FY 2021 Q2)

When comparing the scores to Q1 2021 specifically, the only loss was a 1 percentage point drop in the satisfaction question (69 to 68 percent). The Confidence, Efficiency, and Employee questions had no change (61, 62, and 66 percent). The Ease, Transparency, and Effectiveness questions increased by 1, 2, and 3 percentage points respectively (63 to 64, 76 to 78, and 61 to 64 percent).

Comparing the scores to the same quarter of the previous year (Q2 of 2020), both the Confidence and Employee questions dropped by 1 percentage point (62 to 61, and 67 to 66 percent). The Satisfaction, Ease, and Efficiency questions had no change (68, 64, and 62 percent). The Effectiveness and Transparency questions rose by 2 and 3 percentage points respectively (62 to 64 and 75 to 78 percent).

Of the seven questions, the Confidence, Effectiveness, and Efficiency questions generally score the lowest. This may indicate that the time it takes to complete tasks and User’s ability to complete said tasks may negatively effect their confidence in the site. The Transparency question consistently scores the highest for both the desktop and mobile surveys, which suggests that there are few issues accessing the site from various devices.

USAJOBS Mobile (Summary FY 2021 Q2)

Comparing the scores to Q1 of 2021, the Ease question had the largest drop of 3 percentage points (77 to 74 percent). The Satisfaction, Efficiency, Transparency, and Employee questions all had a 2 percentage point decrease (81 to 79, 74 to 72, 87 to 85, and 79 to 77 percent). There was no change in the Confidence question score (75 percent), and an increase of 1 percentage point in the Effectiveness question score (75 to 76 percent).

Comparing the scores to the same quarter of the previous year (Q2 of 2020), both the Ease and Employee questions dropped by 3 percentage points (77 to 74 and 80 to 77 percent). The Satisfaction question score dropped by 2 percentage points (81 to 79 percent), and the Confidence ad Efficiency questions both dropped by 1 percentage point (76 to 75 and 73 to 72 percent). There was no change in the Effectiveness and Transparency question scores (76 and 85 percent).

The Efficiency question consistently scores the lowest in this survey. This suggests the amount of time to complete tasks may be an area of potential improvement for the site as a whole. Like the desktop survey, the Transparency question consistently scores highest for this survey.

Service details

Q3 2023

What we learned from this quarter's data:

As reported previously, OPM’s survey provider migrated from a legacy platform to a FedRAMP platform in Q2 FY 2023, which led to sizable gaps in data for the desktop and mobile surveys. While OPM collected complete results for both surveys in Q3 FY 2023, it is still possible that there are lingering effects on this quarter’s results related to the migration due to initial differences in survey questions and scoring protocol which were subsequently reverted to what was previously shown on the legacy platform.

In Q3 FY 2023, the overall satisfaction scores for both the USAJOBS desktop and mobile applications decreased compared to Q1 FY 2023 results. The overall satisfaction score for USAJOBS Desktop decreased from 3.95 to 3.91, while the overall satisfaction score for USAJOBS Mobile decreased from 4.27 to 4.14 from Q1 FY 2023 to Q3 FY 2023.

The scores for the drivers of satisfaction (effectiveness, ease, efficiency, transparency, and website helpfulness) in both the desktop and mobile surveys also decreased almost across the board from Q1 FY 2023 to Q3 FY 2023, the one exception being a 0.05 point increase in desktop transparency. On average, desktop scores fell .03 points, while the average decrease in mobile drivers of satisfaction scores dropped by .13 points. Further study is required to determine whether these decreases are lingering effects of the survey platform migration, as there are no obvious drivers of such change such as major system issues.

While overall satisfaction with the USAJOBS help desk fell 0.13 points from Q2 to Q3 FY 2023, most of the drivers of satisfaction scores increased significantly: trust (2.86 to 3.30), effectiveness (2.67 to 3.39), ease (2.71 to 3.34), efficiency (2.84 to 3.16), and employee helpfulness (2.96 to 3.13). On the other hand, the score for transparency fell 0.54 points (3.27 to 2.73).

In one year of data collection, scores for the help desk survey, which has a small sample size, have been quite volatile, with significant variations in either direction from quarter to quarter.

Service details

Q2 2023

What we learned from this quarter's data:

In Q2 FY 2023, all seven scores collected in the USAJOBS Help Desk Feedback survey decreased compared to Q1 FY 2023, with all but overall satisfaction falling below the Q4 FY 2022 results. Compared to Q1 FY 2023, scores declined between 0.04 points for transparency (3.31 to 3.27) to 0.17 points for trust or confidence (3.03 to 2.86). Overall satisfaction fell .11 points to 2.91.

Continuing a trend set in past quarters, individual responses for each question tended to be very positive or very negative, with fewer middle-range responses. This division in responses is likely caused by respondents contacting the help desk when they encounter a problem and then responding to the survey only after the help desk either resolves or cannot resolve their issue.

To improve customer satisfaction, the USAJOBS help desk staff collects responses in real time and contacts submitters who were not satisfied. Currently, it is unclear whether satisfaction is trending positively or negatively. After three quarters of data collection, average results generally seem to be fairly steady around a neutral response, with variations in either direction from quarter to quarter.

OPM is unable to provide complete Q2 FY 2023 survey results for either the USAJOBS Desktop or USAJOBS Mobile surveys. During Q2 FY 2023, OPM’s survey provider migrated from a legacy to a FedRAMP platform in the middle of the reporting period. Due to this migration, there are sizable gaps in the data collected. OPM has worked with the vendor to address the issues encountered during migration, and expects to be able to report complete and accurate results in Q3 FY 2023.

Service details

Q1 2023

What we learned from this quarter's data:

In Q1 FY 2023, the overall satisfaction scores for both the USAJOBS desktop and mobile applications increased slightly compared to Q4 FY 2022 results. Continuing a trend of slight increases beginning in Q2 FY 2022, the overall satisfaction score for USAJOBS Desktop increased from 3.89 to 3.95, while the overall satisfaction score for USAJOBS Mobile rose from 4.25 to 4.27 from Q4 FY 2022 to Q1 FY 2023.

The scores for the drivers of satisfaction (effectiveness, ease, efficiency, transparency, and website helpfulness) in both the desktop and mobile surveys also increased slightly from Q4 FY 2022 to Q1 FY 2023. Individual scores for these questions, as well as the confidence/trust question, increased by an average of 0.05 for the desktop survey and 0.04 for the mobile survey. While further research is needed, these positive trends may be the result of changes made by OPM to the site in responses to data collected from customer experiences.

In Q1 FY 2023, six of the seven scores collected in the USAJOBS Help Desk Feedback survey increased compared to Q4 FY 2022, and the overall satisfaction score rose by 0.17 points (2.85 to 3.02). The one exception was transparency, which decreased slightly from 3.35 to 3.31, though this was still the highest scoring question on the survey. Continuing a trend set in past quarters, individual responses for each question tended to be very positive or very negative, with fewer middle-range responses. This division in responses is likely caused by respondents contacting the help desk when they encounter a problem and then responding to the survey only after the help desk either resolves or cannot resolve their issue. As this is the second quarter that data has been collected for this survey, OPM requires additional data to determine whether this quarter’s increases represent a broader trend of improvement. To improve customer satisfaction, the USAJOBS help desk staff is collecting responses in real time and contacting submitters who were not satisfied to drive improvements to satisfaction.

Service details

Q4 2022

What we learned from this quarter's data:

In Q4 of FY 2022, the overall satisfaction scores for both the USAJOBS desktop and mobile applications stayed consistent with the scores in Q3 of FY 2022 as well as those of FY 2021. For the desktop survey, the overall satisfaction score rose from 3.88 to 3.89 from Q3 to Q4. For the mobile application, the score rose from 4.23 to 4.25 from Q3 to Q4. Changes to scores were minimal.

The scores for the drivers of satisfaction (effectiveness, ease, efficiency, transparency, and website helpfulness) in both the desktop and mobile surveys also stayed within a consistent range from Q3 to Q4 of FY 2022. Individual scores for these questions, as well as the confidence/trust question, mostly showed no change or small increases from FY 2022 Q3 to Q4. All score changes were minimal (less than 0.05).

As the USAJOBS Help Desk Feedback survey was deployed for the first time this quarter, there is no past performance for which to compare scores. In Q4, average scores varied from 2.69 (effectiveness) to 3.35 (transparency). Generally, individual responses for each question tended to be very positive or very negative, with fewer middle-range responses. This may be caused by the fact that respondents tend to contact the help desk when they encounter a problem and will then respond to the survey after the help desk either resolves or cannot resolve their issue. The USAJOBS help desk staff have been collecting responses in real time and contacting submitters who were not satisfied, which may positively impact scores in the future.

Service details