Using Goal Achievement as an Outcome Measure Following a Gero-Oncology Educational Program for Oncology Nurses: The GrOW Curriculum

Cancer is a disease that increases in incidence with age, and the number of older adults in the United States is growing; therefore, more older adults are being diagnosed with cancer (American Cancer Society, 2021; Institute of Medicine [IOM], 2008). Fifty-seven percent of cancer diagnoses occur in those aged 65 years or older, and because of the aging population, this number is expected to grow to almost 70% by 2030 (National Cancer Institute [NCI], Surveillance, Epidemiology and End Results Program, 2013; Smith et al., 2009). These demographics show that most oncology healthcare professionals are caring for an older patient population. Concurrent with the growth of the geriatric oncology population, there is a projected workforce shortage of oncology and gerontology professionals to care for these patients. Recommendations from the IOM report Retooling for an Aging America: Building the Health Care Workforce state that “to meet the health care needs of the next generation of older adults, the geriatric competence of the entire workforce needs to be enhanced [. . .] innovative models need to be developed and implemented” (IOM, 2008, p. 6). The report found that a low percentage of healthcare workers have geriatric training to prepare them to care for older adults. It also found that all members of the healthcare team will need to have knowledge in caring for older adults to meet this need. Nurses are key in caring for older adults, and few of them have sufficient training in geriatrics. Older adults can develop vulnerabilities that put them at higher risk when receiving cancer care (American Geriatrics Society Expert Panel on the Care of Older Adults With Multimorbidity, 2012). The care of older adults can be complicated by the presence of comorbid illnesses, declines in organ function or physiologic reserves, and increased need for assistance with daily function. Challenges in managing multimorbidity are present because greater than 50% of older adults have three or more chronic diseases (American Geriatrics Society Expert Panel on the Care of Older Adults With Multimorbidity, 2012). Older adults with cancer are at an increased risk for complications and serious chemotherapy toxicity, and a predictive model suggests multiple factors, including those identified in a comprehensive geriatric assessment, may increase their risk (Dees et al., 2000; Hurria et al., 2011). Oncology professionals may not be aware of these factors and geriatric syndromes that affect older adults with cancer. The ability to incorporate geriatric knowledge into care allows patients and their caregivers to better balance the risks and benefits of cancer therapy. Fried et al. (2002) found that patient preferences depended on functional and cognitive outcomes of treatment. They stressed the importance of honoring preferences of older adults with serious illnesses. The goal of understanding and applying the principles of geriatric education to an aging oncology population is to preserve the function and well-being of older adults with cancer.

Based on the need to educate oncology nurses in gerontology, a grant was submitted to the National Institutes of Health (NIH) to support educating oncology nurses in the care of older adults (No. R25CA183723-01A1; former principal investigator [PI]: Hurria; current PIs: Loscalzo and Dale). The grant involved creating an educational curriculum for nurses and dissemination via four annual workshops (Burhenn et al., 2020). The grant aimed to educate and train oncology nurses in the concepts of gerontology to improve oncology care for older adults. The Geriatric Oncology Workshop (GrOW) curriculum was developed by the current authors and attended by 387 nurses over four annual courses. A previous article (Burhenn et al., 2020) describes the overall project and the GrOW curriculum development and methods, along with the outcome data related to the knowledge gained and program evaluations for all four annual workshops.

As an additional outcome, nurses worked in teams and were asked to develop goals based on the knowledge gained from the workshops that they would like to achieve once they returned to their practice settings. The nurses were specifically asked to take the knowledge gained from this newly developed curriculum and to develop SMART (specific, measurable, achievable, relevant, timely) goals at the end of the educational workshop. These goals were then assessed at 6, 12, and 18 months postworkshop to determine if the goals were met using the knowledge gained. The purpose of this descriptive qualitative analysis was to measure the outcomes and barriers of achieving goals set at a workshop that taught oncology nurses key gerontology concepts. These goals were tracked longitudinally for goal achievement as measured outcomes.

Methods

Methodologic Approach

The curriculum development and application process for attending the program included the six core principles important to adult learning (Knowles et al., 2011). These included the need to know, understanding the learners’ self-concept (autonomous and self-directed), the role of the learners’ past experiences working with older adults, readiness to learn, orientation to learning (workshop designed using case-based learning that was problem-centered and contextual to the participants needs), and motivation to learn. The application process provided information related to interest in attending the course and perceived need for learning this new information. Self-determination theory (SDT) and achievement goal theory (AGT) provided a conceptual framework for understanding what motivates people to achieve goals (Senko et al., 2012; Urdan & Kaplan, 2020). Similar to other motivation theorists, the developers of SDT and AGT focus on goal-related efficacy and the psychological need for competence, relatedness, and autonomy (Deci & Ryan, 2000; Standage et al., 2003). Building the desire to gain knowledge in a specific area involves intrinsic and extrinsic motivation (Standage et al., 2003). Therefore, the act of achieving goals has multiple components related to the person who develops the goals and attempts to complete them within the environment of their individual settings.

The educational program provided the knowledge and follow-up to support a sense of competence in nurses’ newly learned skills to manage older adults with cancer. Relating these new concepts to their specific practices and allowing them to develop goals that would implement that knowledge into their practice was important. Ensuring that each participant had a sense of autonomy and the ability to function independently was a necessary starting point.

Program participants completed a competitive application that included letters from their supervisors verifying support of the application and acknowledging that the participants would develop goals to implement once they returned to their practice settings following the workshop. The application process required participants to provide information about their background and perceived need to learn this new knowledge. They were asked to describe how they planned to disseminate their new knowledge as part of the application. This allowed the authors of the current article to examine participants’ motivation and willingness to apply this new knowledge. They recognized the need to develop geriatric-specific skills and knowledge to improve oncology care for this growing population of older adults with cancer in their individual practice settings. According to Knowles et al. (2011), the motivation to learn comes when people perceive that learning will assist them in real-life situations. As a growth-oriented activity, goals allowed participants to focus and integrate their new knowledge into their practice settings, as well as influence organizational change.

The process of goal-setting within an educational program has been used over time to evaluate the effects of education on the participant and commitment to implementation (Chou et al., 2011; Kiresuk & Lund, 1994; Shankar et al., 2020). Goal achievement follow-up over 18 months was used to evaluate the effects of the educational program and the implementation of this new knowledge into practice in participants’ practice settings.

Participants and Setting

The GrOW curriculum included four workshops organized by City of Hope and held annually in southern California from 2016 to 2019. Participants were recruited in teams of three (direct care nurse, nurse manager, and nurse educator) from oncology settings across the United States yearly. Settings included NCI-designated cancer centers, community settings, and private oncology settings. Smaller teams were occasionally accepted on a case-by-case basis as judged necessary because of varying setting sizes and the inability to send three staff members at one time. A detailed description of the individual participants is provided in the curriculum development and overall results article (Burhenn et al., 2020). Consistent with published NIH guidance (2012), research conducted in commonly accepted educational settings is exempt as human subject data and does not require consent or institutional review approval.

On day 1 of the workshop, a lecture was given on how to develop SMART goals based on Doran’s (1981) model. The SMART acronym represents a set of five criteria for goals that are specific, measurable, achievable, relevant, and timely. This provided the framework for goal development (Conzemius & O’Neill, 2006). Throughout the workshop, specific time was set aside to allow teams to refine their goals and integrate newly learned knowledge into their plans. Faculty provided support and mentorship during these times. The SMART framework ensured each team’s goals would be efficient, effective, and more likely to be achieved (Doran, 1981).

Procedures and Data Management

Following each educational workshop, three goals were provided by the participant teams. The program follow-up plan is shown in Figure 1. The goals were organized into themes then grouped into categories. The categories were then used to group goals for all subsequent workshops. Three independent reviews were done by authors A.H., P.B., and D.E. for workshops 1 and 2 and by authors C.U., D.E., and P.B. for workshops 3 and 4. Any differences were discussed between the raters until consensus was achieved. Each goal could have multiple categories; therefore, the number of categories applied to each goal could be more than one per goal. As an example of the categorization process, consider the following stated goal: Develop health teaching documents related to normal aging and the relationship to cancer treatments, and provide gerontology education sessions to all participating nurses within six months. This goal was categorized as resource development and professional education.

Participants self-reported on their goal achievement and related barriers and facilitators by completing a goal-tracking and barriers form sent to each team by email at each follow-up time point. At minimum, three attempts were made to collect data via email and telephone follow-ups if forms were not returned by a set deadline. Workshops 1 and 2 study data were collected via email into Microsoft® Excel® files and later entered into REDCap for final analysis by the project statistician. Beginning with workshop 3, the form was automated via REDCap, and a link was sent electronically to the participants for self-completion at each time point.

Goal achievement was reported as (a) completed, meaning the goal had fully been implemented; (b) in process, which was used for teams actively working on the goal at the time of report; (c) stalled, which indicated that the goal was still planning to be achieved, but no action was being taken on the goal at this time; (d) stopped or canceled, which implied that the goal had been started but then had been dropped; or (e) never started, which applied if no action was ever taken.

Barriers experienced by teams were documented in two ways. First, for workshops 1 and 2, barriers to goal achievements were gathered via an open-ended question within the emailed goal analysis form. For workshops 3 and 4, the barriers gathered from workshops 1 and 2 were provided in a checkbox section of the REDCap form. A miscellaneous section was also added for open-ended responses and identified other barriers, such as technology limitations (e.g., electronic health records), staff attrition, infrastructure limitations, lack of geriatric patients, or revision to goal needed.

To further support the participants, conference calls with faculty and other participants in attendance were scheduled monthly. These calls continued until eight months after workshop 4. During the calls, participants shared their goal achievement status, explained barriers and challenges experienced during implementation of their goals, and received feedback from faculty and other participants. At each follow-up time point, the goal-achievement forms were reviewed by the investigators, and any clarifications needed were discussed with each team via email or a telephone call.

Goal categories, barriers, and baseline and 6-, 12-, and 18-month goal-achievement follow-up results were summarized in frequencies and proportions (completed, in-process, stalled, stopped or canceled, or never started). At each time point, goal achievement was calculated as the number of goals with specific achievement status divided by the total number of goals. Barriers were summarized in frequencies and proportions for overall postworkshop follow-up. Data were analyzed using SAS, version 9.4.

Findings

Oncology nurses working in a variety of oncology settings from across the United States and two international sites (Canada and Australia) participated in the GrOW curriculum. For the four workshops (N = 140), most attendees were from NCI-designated cancer centers (n = 52, 37%) or community or general hospitals (n = 34, 24%). Most were three-person teams from the same institution (n = 99, 71%). Participants were asked if they had prior training in geriatrics; 37% (n = 143) said yes, 61% (n = 236) said no, and 2% (n = 8) did not disclose. Table 1 provides workshop demographics by year, team size, and number of goals followed.

Goal Categories

The four workshops totaled 140 teams with 387 participants. One hundred thirty-six teams provided a minimum of three goals each, and four teams provided four goals at the end of the workshop. For all four workshops, there were a total of 424 goals (102 in workshop 1, 106 in workshop 2, 114 in workshop 3, and 102 in workshop 4). The number of goal categories applied to all the goals for all four workshops was 761, and those categories are detailed in Table 2.

The top three categories for the goals overall four years focused on professional education (n = 172, 23%), structure/team building (n = 117, 15%), and resource development (n = 74, 10%).

An example of a professional education goal is, “By the fall, we will have adapted our unit education to include gero-oncology specific education to be included with new employee orientation” (Salt Lake City, Nevada, 2016). Another is to “develop and implement an e-learning education course on geriatric patient vulnerabilities for our inpatient and outpatient nurses for our hospital system in six months” (Oshkosh, Wisconsin, 2017).

Structure/team building goals focused on developing a team or building the structure to care for older adults. An example is “creating an interdisciplinary survivorship subcommittee of stakeholders to identify best gero-oncology screening/assessment tools to develop a referral algorithm by June 30, 2019” (Newark, Delaware, 2016). Another is to “start interdisciplinary team rounds every week on Wednesdays beginning next quarter” (Los Angeles, California, 2016).

Resource development goals included goals such as, “Create a list of community resources for our geriatric patients/find local resources already in place and disperse information on paper as well as on website within 12 months” (Germantown, Tennessee, 2019). Another example is, “We will work to ensure our discharge instructions, appointment reminders, patient education cards, and videos are age-friendly and appropriate for vision- and hearing-impaired within six months” (New York, New York, 2016).

Goal Achievement

The overall data show that, by 18 months for workshops 1–4, more than 70% of the goals were in process or achieved. Table 3 shows results for each follow-up at 6, 12, and 18 months postworkshop. The number of goals that had no follow-up are also listed.

By 18 months, for the top three categories, the results show the professional education category had a total of 172 goals, of which 125 were in process or completed; 21 were stalled; and 26 were never started, stopped, or canceled. For the structure/team building category there was a total of 117 goals, of which 75 were in process or completed; 17 were stalled; and 25 were never started, stopped, or canceled. For the category of resource development, a total of 74 goals were attempted, of which 53 were in process or completed; 9 were stalled; and 12 were never started, stopped, or canceled.

Barriers

Table 4 provides an overview of the number of barriers and percentage for each workshop. The top barriers for each workshop varied slightly each year. Consistent barriers each year included time constraints, lack of institutional support, staffing shortages, or staff attrition.

Miscellaneous barriers included difficulty for infusion nurses to attend educational courses, challenge with interdepartmental and interprofessional communication and collaboration, challenges with support and buy-in from various cancer center providers, rapidly expanding oncology programs requiring different resources, lack of physician support, and restructuring within the cancer setting changing priorities toward geriatric resource implementation.

Discussion

This study of goal achievement following an educational curriculum for oncology nurses demonstrated that setting goals during a conference and having regular follow-up may have supported goal achievement. In an effort to implement evidence-based practices, goal achievement has been used in nursing and other disciplines to affect care. Grant et al. (2017) developed a similar model to improve survivorship care in oncology settings by providing specialty training and using goal achievement to measure implementation of new knowledge into practice. Although the nurses attended individually, multiple participants from the same settings were recommended during the four-year time span to provide enhanced support for goal efforts. Goal achievement in this curriculum was similarly measured from baseline to 6, 12, and 18 months postworkshop. At 18 months after the workshops were finished, 28%–36% completed their goals and 27%–41% were in process. Professional education was also a top category in their goal categories, recognizing the gap in knowledge and the desire to increase knowledge in their colleagues to improve care.

A study by Chou et al. (2011) identified participant outcomes secondary to an educational program using a goal-commitment framework to improve knowledge of guidelines and whether best practice models were implemented into practice. Education included a teamwork format, quality improvement activities, methods to improve coordination among departments, and gathered participation and performance feedback to estimate best practice activities. They found that participants who had input into the guideline implementation process and involvement in quality improvement projects were more motivated to follow through with implementation of their goals. They also found that, when participants felt teamwork was present, agreement, knowledge improvement, and use of best practice recommendations significantly improved (Chou et al., 2011).

Goal achievement for each workshop in the GrOW curriculum showed that more than 70% were in process or completed by 18 months. Working within teams provided support for nurses when barriers affected their efforts to implement new knowledge into their practice settings. The number of goals that were stopped, stalled, or canceled were barriers related to team attrition, staffing shortages, time constraints, lack of institutional support, and lack of geriatric knowledge and training. Barriers to goal implementation, such as lack of interest and changes in setting priorities, may also have attributed to their inability to achieve their goals. For workshop 4, 18-month follow-up took place in August 2020. The COVID-19 pandemic was affecting healthcare facilities at that time and became a major barrier to getting goal-achievement feedback and hindered participants’ ability to continue their goal implementation because of changes in patient and staff interactions and institutional priorities. Sustainability is a challenge with attrition and turnover, and continues to be a major concern for building new geriatric-focused practices that can be maintained over time. The findings noted that staff shortages and attrition were common barriers to goal implementation.

Team size may have contributed to goal achievement. Having more people who share goals for practice settings can continue the efforts when role changes or attrition occurs. A major challenge for nurse-led implementation of new knowledge is having limited or no institutional support from administrators and physicians. One institution that had sent two different teams realized that working together on the same goals would allow them the flexibility and bandwidth they needed to achieve their goals and keep moving forward. In addition, accountability to the GrOW curriculum team, by establishing an agreement as part of their application process for required longitudinal follow-up, provided motivation for teams to meet their goals. Although it took multiple efforts to get data returned for some settings, the active interaction and mentoring of the teams contributed to their successful implementation of new geriatric knowledge into their settings.

Limitations

There were limitations in the study. Although 31 of 50 states in the United States sent teams to attend, this program was only offered in the Los Angeles area of California, which may have restricted access for some. A small travel stipend was provided to mitigate this, but reimbursement for nurses in some settings is limited. Participants were oncology nurses who self-selected and were motivated to attend an educational program to gain geriatric knowledge to improve their practice, which may have affected the results. The determinations of goal completion were self-reported, and the authors were unable to validate if the goals had been met. Therefore, it is possible that goal achievement could have been overreported.

It is important to note that, for workshops 1 and 2, barrier reporting was an open-ended comment box, and participants free-texted their barriers for each goal as compared to workshops 3 and 4, where participants were able to electronically select the barriers on their electronic form and had a fill-in space for other barriers if needed. This may have improved the ease of documenting barriers. Using an electronic method, such as REDCap, to gather follow-up data may have made reporting barriers easier and may explain why a greater number of barriers were reported for workshops 3 and 4.

The aim of this article was to evaluate the effects the GrOW curriculum had on nursing practice for older adults with cancer; goals were not followed past 18 months postworkshop, so the impact beyond 18 months is not known. A three-person team was preferred, but not all institutions were able to send a three-person team. Teams with fewer members may have had limited support to implement their goals. Forms and reminders were emailed, and many follow-up requests were needed to secure data collection. Support was offered on monthly telephone calls; however, because of differing time zones and work responsibilities, some teams had difficulty fully accessing this support. Most attendees were from NCI-designated cancer centers, which may have more support for goal implementation because of size and staff, and the results may not fully represent less-resourced environments. Although one lecture in the workshop focused on securing support for goals from management, many other barriers were uncovered during the follow-up months. Lack of institutional support and lack of staff made implementation for nurses with multiple priorities more challenging. Future programs should address these barriers during the workshop to prepare teams to anticipate and hopefully minimize barriers to goal implementation.

Implications for Nursing

Nurses play a major role in successful implementation of change to improve patient care. Using goals to initiate the implementation of new knowledge into practice after educational programs may help to increase the likelihood of successful integration into the practice setting. Working as a team consisting of management, an education-focused nurse, and a direct patient care role improves the ability to achieve goals and change practice.

Future research aimed at implementation of new knowledge into practice that includes team and individual impact within SDT and AGT frameworks and the changes that occur in practice would be important work. In addition, studies focusing on barriers to achieving goals in the clinical setting and interventions to overcome those barriers would be key in this area of integrating new knowledge.

Conclusion

In the current study, oncology nurses were likely to achieve their goals when they created teams with management, education-focused, and direct patient care expertise; attended an educational program; and developed SMART goals with interactive, longitudinal follow-up. This has implications for other educational programs that nurses may attend. Nurses play an important role in improving care for patients and implementing new knowledge into practice. Using a structured approach to goal development, such as SMART goals, was essential to set measurable and timely goals initiatives. In addition, requiring a team of nurses that includes management, education-focused, and direct patient care staff to support goal implementation helped to provide broad nursing skills and expertise to support goal achievement. Institutional support and working with a team that can share efforts to implement goals into practice can further enhance success. Encouraging participants to set goals that define how newly acquired knowledge will be used in their practice settings after attending an educational program can assist nurses in successfully implementing change that will improve oncology nursing practice and provide optimal care to older adults with cancer. 

About the Author(s)

Denice Economou, PhD, CNS, was, at the time of writing, a senior research specialist, Peggy S. Burhenn, MS, RN-BC, CNS, AOCNS^®, OCN^®, and Carolina Uranga, MSN, RN-BC, AGCNS-BC, OCN^®, are clinical nurse specialists, Heeyoung Kim, MPH, is a biostatistician, Can-Lan Sun, PhD, is an associate research professor, Vani Katheria, MS, is a scientific program manager, Dale Mitani, BA, is a senior application developer, Matthew J. Loscalzo, LCSW, is the Liliane Elkins Professor in Supportive Care Programs, administrative director of the Sheri and Les Biller Patient and Family Resource Center, executive director of the Department of Supportive Care Medicine, and a professor in the Department of Population Sciences, Arti Hurria, MD, was, at the time of writing, director of the Center for Cancer and Aging, and William Dale, MD, PhD, is the Arthur M. Coppola Chair of Supportive Care Medicine and the director of the Center for Cancer and Aging, all at City of Hope National Medical Center in Duarte, CA. No financial relationships to disclose. Economou, Burhenn, Uranga, Sun, Hurria, and Dale contributed to the conceptualization and design. Economou, Burhenn, Uranga, Kim, Sun, Katheria, Mitani, and Hurria completed the data collection. Economou, Kim, and Sun provided statistical support. Economou, Burhenn, Kim, Sun, and Dale provided the analysis. Economou, Burhenn, Uranga, Kim, Sun, Katheria, Loscalzo, Hurria, and Dale contributed to the manuscript preparation. Economou can be reached at deconomou@sonnet.ucla.edu, with copy to ONFEditor@ons.org. (Submitted January 2021. Accepted August 4, 2021.)