Article

Training Oncology Nurses to Use Remote Symptom Support Protocols: A Retrospective Pre-/Post-Study

Dawn Stacey

Myriam Skrutkowski


Meg Carley

Erin Kolari

Tara Shaw


Barbara Ballantyne

oncology, nursing, symptoms, training, knowledge, confidence
ONF 2015, 42(2), 174-182. DOI: 10.1188/15.ONF.174-182

Purpose/Objectives: To evaluate the impact of training on nurses’ satisfaction and perceived confidence using symptom protocols for remotely supporting patients undergoing cancer treatment.

Design: Retrospective pre-/post-study guided by the Knowledge-to-Action Framework.

Setting: Interactive workshops at three ambulatory oncology programs in Canada.

Sample: 107 RNs who provide remote support to patients with cancer.

Methods: Workshops included didactic presentation, role play with protocols, and group discussion. Post-training, a survey measured satisfaction with training and retrospective pre-/post-perceived confidence in the ability to provide symptom support using protocols. One-tailed, paired t-tests measured change.

Main Research Variables: Satisfaction with the workshop and perceived confidence in the ability to provide symptom support and use protocols.

Findings: Twenty-two workshops, 30–60 minutes each, were conducted with 107 participants. Ninety completed the survey. Compared to preworkshop, postworkshop nurses had improved self-confidence to assess, triage, and guide patients in self-care for cancer treatment–related symptoms, and use protocols to facilitate symptom assessment, triage, and care. Workshops were rated as easy to understand, comprehensive, and provided new information on remote symptom management. Some specified that the workshop did not provide enough time for role play, but most said they would recommend it to others.

Conclusions: The workshop increased nurses’ perceived confidence with providing remote symptom support and was well received.

Implications for Nursing: Subsequent workshops should ensure adequate time for role play to enhance nurses’ skills in using protocols and documenting symptom support.

Jump to a section

    Patients and their family members contact healthcare providers at ambulatory oncology programs for guidance in symptom management. In many oncology programs, nurses are the first line of contact for assessing the severity of symptoms, triaging to the level of healthcare services required, and, if appropriate, guiding the patient in self-management (Macartney, Stacey, Carley, & Harrison, 2012). To enhance the quality of remote support, guidelines recommend that nurses are trained to provide remote support, follow protocols, document their calls, and monitor outcomes of remote healthcare services (Canadian Nurses Association, 2007).

    As part of a pan-Canadian initiative, a set of 13 evidence-based symptom protocols was developed to translate evidence from clinical practice guidelines into user-friendly tools relevant for use in clinical practice (Stacey, Macartney, Carley, Harrison, & the Pan-Canadian Oncology Symptom Triage and Remote Support Group [COSTaRS], 2013). The 13 symptoms included anxiety, bleeding, breathlessness, constipation, depression, diarrhea, fatigue and tiredness, fever with neutropenia, loss of appetite, mouth sores, nausea and vomiting, peripheral neuropathy, and skin reaction. No clinical practice guidelines were available for dysuria or cognitive changes, and no protocol was developed for pain, given the number of protocols already present within organizations. Protocols were developed by a pan-Canadian committee of researchers, information systems specialists, methodologists, library scientists, advanced practice nurses, and nurse leaders using a systematic process guided by the CAN-IMPLEMENT© methodology, a guideline adaptation and implementation planning framework (Harrison et al., 2013; Harrison, van den Hoek, & the Canadian Guideline Adaptation Study Group, 2012). Protocol development involved a systematic review to identify guidelines; consensus on the two-page template designed to meet rigor score criteria for the Appraisal of Guidelines, Research, and Evaluation II (AGREE II) instrument (Brouwers, Kho, et al., 2010); and review by oncology nurses in a range of positions (e.g., direct care, educators, advanced practice nurses, managers, researchers). Finally, each symptom protocol was written using plain language to facilitate oncology nurses’ ability to support patients experiencing cancer treatment–related symptoms. Elements included in the protocols are criteria to assess symptom severity and triage based on highest severity, review of medications and self-management strategies for symptom management, and space to document an agreed-upon plan. The symptom protocols are publicly available in English and French on the Knowledge Translation Canada or Canadian Association of Nurses in Oncology websites (www.ktcanada.ohri.ca/costars or www.cano-acio.ca/triage-remote-protocols).

    However, an assessment of factors influencing use of these new symptom protocols in clinical practice revealed that only 28% of nurses used any protocols and 73% agreed that they need to enhance their knowledge about using symptom protocols for providing remote support (Stacey, Carley, Ballantyne, et al., 2014). These findings are consistent with other studies indicating that nurses need educational opportunities to learn how to use symptom protocols (Macartney et al., 2012; Stacey, Bakker, Green, Zanchetta, & Conlon, 2007). To increase use of these protocols, effective interventions (e.g., educational programs) need to be selected to overcome factors identified to interfere with their uptake in clinical practice (Baker et al., 2010).

    An environmental scan of remote symptom support training within Canadian ambulatory oncology programs revealed that few programs have formal training and none were rigorously evaluated (Stacey, Carley, Kohli, et al., 2014). Of the seven programs that shared materials, two included objectives focused on using symptom management guidelines and four others discussed use of symptom management guidelines during the training. Approaches to training included one or more of the following: passive dissemination of printed educational materials, formal classroom training, online tutorials, mentoring, and performance feedback. Previous research has shown that effective interventions for enhancing uptake of knowledge and skills in clinical practice include educational meetings combined with educational outreach and/or performance feedback; printed education alone was found to be less effective (Grimshaw, Eccles, Lavis, Hill, & Squires, 2012).

    To identify relevant educational programs, a literature search was conducted using key words for five concepts (oncology nursing, telephone support, protocols and practice guidelines, training, and evaluation) in MEDLINE®, EMBASE, CINAHL®, and PsycINFO® from January 1996 to July 2014. No citations were identified. A series of follow-up searches with varying combinations of the five key concepts revealed three articles (Knowles et al., 2008; Pasacreta, Kenefick, & McCorkle, 2008; Stacey, Chambers, Jacobsen, & Dunn, 2008). A pilot study by Knowles et al. (2008) revealed that a four-day educational intervention plus self-directed learning improved oncology nurses’ (N = 67) knowledge and attitudes about managing patients with colorectal cancer. A pre-/post-test study (Stacey et al., 2008) found that an online tutorial (1.5 hours) plus an interactive workshop (three hours) improved healthcare professionals’ (N = 32) knowledge, skills, and confidence in supporting adults making cancer screening and treatment decisions. The third study (Pasacreta et al., 2008) reported that oncology nurses were satisfied with a set of four webcast online presentations on distress management. However, the webcasts were completed by 276, 90, 64, and 64 nurses, respectively.

    The overall aim of the current study was to evaluate the impact of training on nurses’ satisfaction and perceived confidence using the COSTaRS protocols for providing symptom support to patients undergoing cancer treatment. The current study was conducted as part of a larger study to build an effective and sustainable approach for implementing the COSTaRS symptom protocols for oncology nurses to use when providing remote symptom assessment, triage, and guidance in self-management (Stacey et al., 2012). An important feature of the current study was the inclusion of knowledge users on the research team (e.g., managers, advanced practice nurses, staff nurses), which is more likely to produce findings of use to them (Bowen & Graham, 2013).

    Methods

    A retrospective pre-/post-study was guided by the Knowledge-to-Action (KTA) Framework. The pre-/post-study approach has been shown to be valid for assessing educational interventions when the training changes participants’ frame of reference for assessing their attitudes or skills (e.g., confidence with using symptom protocols, providing remote support) (Howard, 1980; Yank, Laurent, Plant, & Lorig, 2013), whereas traditional pre-/post-measures often lead to underestimation of program effects because of response shift bias.

    The KTA Framework has two main sections, knowledge creation at the center and an action cycle. Knowledge creation involves synthesis of individual studies into systematic reviews, with findings presented in knowledge translation tools (Graham et al., 2006). The researchers’ knowledge translation tools were the 13 COSTaRS protocols. In the current study, the focus was on the action cycle of the KTA Framework. For moving knowledge translation tools into clinical practice, interventions or strategies are required to overcome identified barriers. Ethics approval for the study was obtained from the Ottawa Health Science Network Research Ethics Board (20120388-01H), from the University of Ottawa Research Ethics Board (A 07-12-02), and at each of the three participating centers.

    The current study took place in three Canadian ambulatory oncology programs within different provincial healthcare systems (see Table 1). One oncology program had a main location with one single outreach clinic (site 1: Nova Scotia Cancer Centre, Capital Health, Halifax, Nova Scotia); another had two main locations (site 2: Montreal General Hospital, Quebec), and the third had one main location with 14 outreach clinics (site 3: Health Sciences North, Sudbury, Ontario). For patients receiving chemotherapy at site 1, consultations with oncologists occurred within the main oncology program and the oncologists took turns traveling to the outreach center on a regularly scheduled basis. For site 3, patients routinely were seen via telemedicine or returned to the clinic for follow-up and local physicians were trained to assist patients receiving chemotherapy who required urgent medical assessment between oncologist visits. Site 3 provided oncology services to a large area of the province that is not densely populated. Eligible participants were RNs who provided remote nursing services to patients with cancer. All remote nursing services for patients were provided via telephone. Individual demographic characteristics were not collected.

    Intervention

    Workshop content was planned based on an environmental scan of remote support training programs and expertise of some team members who provided continuing education sessions for oncology nurses (Stacey, Carley, Kohli, et al., 2014). By the end of the workshop, participants were expected to be able to (a) describe elements of evidence-informed remote symptom support, (b) explain how to assess and triage patients’ cancer treatment–related symptoms, (c) discuss tailoring symptom self-care to patients’ needs, (d) use evidence- and theory-based protocols/practice guides for patients experiencing cancer treatment–related symptoms, (e) review basic principles and conduct for remote support (e.g., communication skills), and (f) know how to document remote support. The workshops were designed to be provided within the regular workday and lasted 35–60 minutes. Teaching strategies included didactic presentation, role play, and group discussion. The workshop is publicly available at www.ktcanada.ohri.ca/costars.

    The didactic presentation aimed to provide consistency across workshops, but content on how to use protocols within everyday clinical practice and document their use was tailored for each oncology program. Role play using one or two symptom protocols as examples provided participants time to use the protocols within a safe learning environment. Group discussion was used to share learnings from the role play experiences and to tailor implementation into clinical practice. For example, the COSTaRS protocols were being implemented as documentation tools at two sites and as pocket guides at another. Interactive workshops using role play and small group discussions are effective learning strategies (Berkhof, van Rijssen, Schellart, Anema, & van der Beek, 2011; Grimshaw et al., 2012).

    The workshop facilitators were RNs from within the ambulatory oncology programs. These facilitators emphasized that the COSTaRS protocols are a pan-Canadian resource based on the best available evidence, intended to support critical thinking skills as opposed to overriding nurses’ expertise, and are meant to encourage patients in managing their symptoms at home. Workshop participants were encouraged to familiarize themselves with the protocols for faster navigation and better flow; reminded to respond to the individual nature of the callers’ symptoms rather than follow assessment items in rank order; and provided with tips based on experiences using them in practice. The need for concise, consistent, and complete documentation of remote support encounters also was emphasized.

    Outcome Measures

    Informed by a four-level training evaluation model, the researchers’ outcome measures focused on participants’ reactions to the program (i.e., satisfaction) and learning (i.e., confidence) (Kirkpatrick & Kirkpatrick, 2005). According to Kirkpatrick and Kirkpatrick (2005), education programs should be evaluated for participant satisfaction because findings have important consequences on participants’ learning. Satisfaction is a measure of how participants react to the program (Kirkpatrick & Kirkpatrick, 2005). Pre-/post-tests can be used to measure change in knowledge, attitudes, and confidence. For the purpose of this article, self-confidence is defined as a measure of a person’s belief that he or she can succeed (Perry, 2011).

    Before leaving the workshop, participants completed a survey to determine their satisfaction with the workshop and self-rate their feelings of perceived confidence in their ability to provide symptom support and use protocols. The satisfaction survey was comprised of 12 multiple choice questions aimed at assessing whether the workshop achieved the learning objectives, provided clear information, was comprehensive, covered new information, and allocated enough time to the role play exercises, and it asked participants about their overall rating and willingness to recommend it to others. Items in the survey are commonly used to evaluate education sessions (Légaré et al., 2012). Additional open-ended questions asked participants what they liked and what suggestions they had for improving the workshop. The survey was modified by the addition of questions regarding specific objectives of the workshop.

    Participants also were asked to rate their perceived confidence with using the new symptom protocols and confidence with providing remote assessment, triage, and guidance in self-management. This outcome was measured postworkshop but evaluated their perception of their baseline perceived confidence (retrospective) and their confidence postworkshop on a scale of 1 (strongly disagree) to 5 (strongly agree). These survey items had good internal consistency (Cronbach alpha = 0.75). This approach is used to evaluate change in healthcare professionals’ confidence and feeling informed (Bhanji, Gottesman, de Grave, Steinert, & Winer, 2012; Yank et al., 2013).

    Analysis

    Analysis of quantitative data was conducted using SPSS®, version 22.0. Satisfaction survey items were analyzed using univariate descriptive statistics. One-tailed, paired t-tests evaluated change in perceived confidence. Differences between groups in level of perceived confidence in ability to use the COSTaRS protocols preworkshop were analyzed using Kruskal-Wallis one-way analysis of variance. Open-ended questions were analyzed qualitatively using content analysis. Patterns and themes were identified, and frequencies of comments relating to each category were tabulated.

    Results

    Twenty-two workshops were provided (see Table 2). Nurses were encouraged to attend, but attendance was not mandatory. Across the three sites, 107 nurses participated, reaching 90% of eligible nurses. At one site, clerical staff, who are responsible for recording patient messages and assigning messages to the appropriate nurse, attended a workshop. Clerical staff did not complete any of the outcome measures.

    Of the 107 nurses who attended the workshop, 90 (84%) completed the survey at the end of the workshop. Participants rated the workshop as easy to understand (n = 88), just the right amount of information (n = 81), and provided new information (n = 78) (see Table 3). Seventy-two nurses indicated that all six learning objectives were addressed (see Table 4). The majority thought they had enough time to try the protocols in role play (n = 50), but others thought they had not enough (n = 30) or too much time (n = 2). Overall, participants rated it as excellent (n = 40) or good (n = 44) and would recommend it to others (n = 82). Findings overall were similar across the three sites, except that more nurses at site 1 (n = 23) felt they had adequate time for role play than nurses at sites 2 (n = 21) and 3 (n = 6).

    Perceived confidence in participants’ ability to assess, triage, and guide patients in self-care for their cancer treatment–related symptoms and confidence in their ability to use the COSTaRS protocols to facilitate symptom assessment, triage, and care statistically improved postworkshop (see Table 5). This improvement was significant in subanalysis by site. Preworkshop, the baseline perceived confidence in participants’ ability to use the COSTaRS protocols was highest at site 2, where nurses reported using the COSTaRS protocols prior to the workshop, but differences between groups at baseline were not statistically significant (p = 0.1).

    Of 90 participants, 53 provided positive comments, 24 provided comments aimed at enhancing the workshop, and 28 did not provide any comments. Nine themes identified from the set of positive comments were interactive format, learning about protocols, clear and informative, small group size, trainer expertise, application of protocols, comfortable learning environment, role play, and sufficient time (see Figure 1). Many liked the interactive format, as exemplified by one participant who said he or she “enjoyed the interaction about how we use or can use guidelines [and] able to discuss our concerns.” Participants were pleased to have the opportunity to learn more about the protocols and application of the protocols in practice, and felt that the content of the workshop was clear and informative. One participant appreciated the workshop being “focused on doing an assessment more than completing the form.” Some commented on the trainer expertise, specifically that they were “knowledgeable and well versed in use of the tools.” Another participant commented, “It feels more applicable when a coworker who is actually familiar with how the phone call live works and is used to telephone triage as opposed to a staff member who is not used to the calls.” Some commented on the comfort of the learning environment, enjoyed doing role play, and felt they had sufficient time to ask questions.

    Suggestions for improving the workshop were indicated by 24 participants, and 6 participants explicitly stated they had no comments to enhance them. Of the comments aimed at enhancing the workshop, most notable was the suggestion for longer workshops with more time allocated for role play. Some participants would have liked to have had training for all protocols or chosen protocols that commonly are used within their practice. A few participants requested “alternatives to role play,” such as “scenarios may have been more effective,” and expressed concern that “role play creates such ‘performance anxiety.’” Some participants wanted reinforcement sessions with ongoing review or more examples. One participant indicated that, “We could benefit from a more advanced workshop looking at some complex case (real) studies.”

    Discussion

    The current study aimed to evaluate training of nurses in remote symptom support using evidence-informed symptom protocols. To the best of the researchers’ knowledge, this study is the first to evaluate this type of training. Overall, nurses who attended the workshop were exposed to new information on the use of evidence-based COSTaRS protocols developed by a pan-Canadian research team, strategies for providing remote support (telephone), and reinforcement of important communication skills. At the end of the workshop, nurses overall were satisfied with the workshop, felt more confident providing remote symptom support, and felt confident using the COSTaRS protocols. As the workshops progressed, the ideal length for the sessions was 60 minutes, or 45 minutes with fewer nurses (e.g., two or three nurses) to provide more individualized training. Nurses also appreciated the interactive format of the workshop, the opportunity to learn more about how to use the protocols, and thought the workshop was concise. Although some nurses explicitly stated that no changes were necessary, others suggested longer workshops with more time for role play, more focus on the content within the protocols to learn more about managing specific symptoms, and reinforcement sessions to strengthen nurses’ knowledge and skills using the protocols, as well as documenting support provided for symptom clusters. For subsequent workshops, verifying that adequate time is provided for role play is important, given the importance of interactivity in the workshops to enhance uptake of new behaviors (Grimshaw et al., 2012). Before making any other substantive changes (e.g., length, content), the impact of the workshop on nurses’ behaviors and practice outcomes (e.g., symptom management) should be measured (Kirkpatrick & Kirkpatrick, 2005).

    The researchers’ findings are consistent with studies that have evaluated educational meetings or workshops as the main intervention to change behaviors of healthcare professionals. In a review of 81 trials involving 11,000 healthcare professionals, educational sessions were shown to change practice and healthcare outcomes by 2%–15%, with larger changes associated with higher rates of attendance, using a mixture of didactic and interactive strategies and focusing on outcomes perceived as serious (Forsetlund et al., 2009). In the current study, the workshops were well attended, and didactic and interactive strategies were used. Shorter workshops increased the reach to more than 90% of eligible nurses and were likely to have been better integrated into the workday schedule compared to other oncology nurse training programs that were longer (half day to four days) and/or online (Knowles et al., 2008; Pasacreta et al., 2008; Stacey et al., 2008).

    In the study proposal, the researchers anticipated needing educational workshops and case rounds (Stacey et al., 2012). The baseline barriers assessment confirmed the need for training in the use of the symptom protocols, along with other barriers such as protocol length, learning curve to use them, lack of flexibility (e.g., feeling tied to a script), lack of time and high workload on nursing staff, and lack of integration into an electronic health record (Stacey, Carley, Ballantyne, et al., 2014). Training was provided, but participants at the workshops requested reinforcement sessions rather than case rounds as a format for enhancing their knowledge and skills. A systematic review of training in basic communication skills indicated that reinforcement sessions or guidance by supervisors enhanced uptake of communication skills (Barth & Lannen, 2011). In the current study, once nurses had time to use the protocols in practice, they may have had additional questions about the protocols or documentation process. Because those who provided the training workshops were colleagues, they also were available to informally answer questions and support uptake of protocols in clinical practice, providing reinforcement.

    The researchers’ evaluation findings revealed satisfaction with the workshop and higher perceived self-confidence with providing symptom assessment and using symptom protocols. According to Kirkpatrick and Kirkpatrick’s (2005) model for evaluating training programs, the four levels of evaluation are reaction to the program (e.g., satisfaction), learning (e.g., attitude, knowledge, skills), changes in trainees’ related behavior in practice, and results (e.g., patient outcomes, organizational level). Therefore, the researchers’ findings are limited to the first two levels of evaluation and additional research is required to more rigorously evaluate the effectiveness of the training workshop to determine its impact on changing nurses’ behaviors and affecting patient outcomes. Further evaluation also could measure protocol uptake in nurses exposed to the workshop compared to usual practice.

    Limitations and Strengths

    The current study examined satisfaction with training and change in perceived confidence as opposed to actual change in behavior, and outcome measures had limited psychometric properties. The retrospective pre-/post-survey design used to assess the level of the participants’ perceived confidence in using symptom protocols and providing remote symptom support may have overestimated the real increase between pre- and post-survey (Hill & Betz, 2005; Howard, 1980; Howard, Millham, Slaten, & O’Donnell, 1981; Yank et al., 2013). In addition, recall bias and social desirability bias are possible with retrospective pre-/post-survey design because those who provided the training workshops were colleagues. However, given the consistent, statistically significant findings, even for subgroups of participants, actual improved perceived confidence is likely. Third-party observation, regarded as the gold standard for evaluating change in participant behavior (D’Eon, Sadownik, Harrison, & Nation, 2008), was not feasible, and traditional pre-/post-design carries the potential for response shift bias. Another area of strength was that the training intervention was delivered using the same slideshow presentation to ensure consistency across sites. Finally, because the training intervention was evaluated in three different oncology programs located within three different Canadian healthcare systems, the findings are likely generalizable to other oncology programs.

    Implications for Nursing

    Using symptom management protocols has the potential to narrow the know-do gap by providing nurses with user-friendly, evidence-based tools to guide their practice (Brouwers, Stacey, & O’Connor, 2010). The COSTaRS protocols are practice tools that were informed by a synthesis of current, quality-rated clinical practice guidelines (Stacey et al., 2013). However, barriers such as inadequate knowledge, skills, and confidence interfere with nurses using them in their clinical practice.

    The current study evaluated an educational program on remote symptom support using a didactic presentation, role play using the symptom protocols, and group discussion. The 30- to 60-minute workshop was rated positively by 84 participants. In addition, nurses said that it enhanced their perceived confidence with providing symptom management and using the symptom protocols. The workshop presentation is available for free at www.ktcanada.ohri.ca/costars. Oncology nurses can consider how they are integrating evidence from clinical practice guidelines into their approach to managing symptoms in practice, identify their needs for training on remote symptom support, and consider using evidence-informed tools and training materials from the current study individually or within their oncology programs.

    [[{"type":"media","view_mode":"media_original","fid":"19561","attributes":{"alt":"","class":"media-image","height":"187","typeof":"foaf:Image","width":"367"}}]]

    Conclusions

    Nurses exposed to an interactive training workshop felt higher perceived confidence in their ability to use symptom protocols for remotely assessing, triaging, and managing symptoms experienced by patients receiving treatment. The training workshop was positively received, with the main suggestion for enhancing the workshop being ensuring adequate time for role play. Nurses appreciated the workshop being provided by nurses experienced in remote support. Findings from the study should be generalizable to other oncology programs because it was conducted with more than 100 nurses from three different Canadian healthcare systems. Additional studies should evaluate the impact of training on nurses’ behaviors in clinical practice and patient outcomes.

     

    References

    Baker, R., Camosso-Stefinovic, J., Gillies, C., Shaw, E.J., Cheater, F., Flottorp, S., & Robertson, N. (2010). Tailored interventions to overcome identified barriers to change: Effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews, 3, CD005470. doi:10.1002/14651858.CD005470.pub2

    Barth, J., & Lannen, P. (2011). Efficacy of communication skills training courses in oncology: A systematic review and meta-analysis. Annals of Oncology, 22, 1030–1040. doi:10.1093/annonc/mdq441

    Berkhof, M., van Rijssen, H.J., Schellart, A.J., Anema, J.R., & van der Beek, A.J. (2011). Effective training strategies for teaching communication skills to physicians: An overview of systematic reviews. Patient Education and Counseling, 84, 152–162.

    Bhanji, F., Gottesman, R., de Grave, W., Steinert, Y., & Winer, L.R. (2012). The retrospective pre-post: A practical method to evaluate learning from an educational program. Academic Emergency Medicine, 19, 189–194. doi:10.1111/j.1553-2712.2011.01270.x

    Bowen, S., & Graham, I.D. (2013). Integrated knowledge translation. In S. Straus, J. Tetroe, & I.D. Graham (Eds.), Knowledge translation in health care: Moving from evidence to practice (2nd ed., pp. 14–23). Oxford, England: Wiley-Blackwell.

    Brouwers, M.C., Kho, M.E., Browman, G.P., Burgers, J.S., Cluzeau, F., Feder, G., . . . Makarski, J. (2010). Development of the AGREE II, part 2: Assessment of validity of items and tools to support application. Canadian Medical Association Journal, 182, E472–E478.

    Brouwers, M., Stacey, D., & O’Connor, A. (2010). Knowledge creation: Synthesis, tools and products. Canadian Medical Association Journal, 182, E68–E72. doi:10.1503/cmaj.081230

    Canadian Nurses Association. (2007). Telehealth: The role of the nurse. Retrieved from http://cna-aiic.ca/~/media/cna/page-content/pdf-en/ps89_telehealth_e.pdf

    D’Eon, M., Sadownik, L., Harrison, A., & Nation, J. (2008). Using self-assessments to detect workshop success: Do they work? American Journal of Evaluation, 29, 92–98.

    Forsetlund, L., Bjørndal, A., Rashidian, A., Jamtvedt, G., O’Brien, M.A., Wolf, F., . . . Oxman, A.D. (2009). Continuing education meetings and workshops. Cochrane Database of Systematic Reviews, 2, CD003030.

    Graham, I.D., Logan, J., Harrison, M.B., Straus, S.E., Tetroe, J., Caswell, W., & Robinson, N. (2006). Lost in knowledge translation: Time for a map? Journal of Continuing Education in the Health Professions, 26, 13–24.

    Grimshaw, J.M., Eccles, M.P., Lavis, J.N., Hill, S.J., & Squires, J.E. (2012). Knowledge translation of research findings. Implementation Science, 7, 50. doi:10.1186/1748-5908-7-50

    Harrison, M.B., Graham, I.D., van den Hoek, J., Dogherty, E.J., Carley, M.E., & Angus, V. (2013). Guideline adaptation and implementation planning: A prospective observational study. Implementation Science, 8, 49. doi:10.1186/1748-5908-8-49

    Harrison, M.B., van den Hoek, J., & the Canadian Guideline Adaptation Study Group. (2012). CAN-IMPLEMENT©: Guideline adaptation and implementation planning resource. Retrieved from http://cancerview.ca/idc/groups/public/documents/webcontent/can_impleme…

    Hill, L.G., & Betz, D.L. (2005). Revisiting the retrospective pretest. American Journal of Evaluation, 26, 501–517.

    Howard, G.S. (1980). Response-shift bias: A problem in evaluating interventions with pre/post self-reports. Evaluation Review, 4, 93–106.

    Howard, G.S., Millham, J., Slaten, S., & O’Donnell, L. (1981). Influence of subject response style effects on retrospective measures. Applied Psychological Measurement, 5, 89–100. doi:10.1177/014662168100500113

    Kirkpatrick, D.L., & Kirkpatrick, J.D. (2005). Evaluating training programs: The four levels (3rd ed.). San Francisco, CA: Berrett-Koehler Publishers.

    Knowles, G., Hutchison, C., Smith, G., Philp, I.D., McCormick, K., & Preston, E. (2008). Implementation and evaluation of a pilot education programme in colorectal cancer management for nurses in Scotland. Nurse Education Today, 28, 15–23. doi: 10.1016/j.nedt.2007.02.002

    Légaré, F., Politi, M.C., Drolet, R., Desroches, S., Stacey, D., & Bekker, H. (2012). Training health professionals in shared decision-making: An international environmental scan. Patient Education and Counseling, 88, 159–169. doi:10.1016/j.pec.2012.01.002

    Macartney, G., Stacey, D., Carley, M., & Harrison, M.B. (2012). Priorities, barriers and facilitators for remote support of cancer symptoms: A survey of Canadian oncology nurses. Canadian Oncology Nursing Journal, 22(4), 235–247.

    Pasacreta, J.V., Kenefick, A.L., & McCorkle, R. (2008). Managing distress in oncology patients: Description of an innovative online educational program for nurses. Cancer Nursing, 31, 485–490.

    Perry, P. (2011). Concept analysis: Confidence/self-confidence. Nursing Forum, 46, 218–230. doi:10.1111/j.1744-6198.2011.00230.x

    Stacey, D., Bakker, D., Ballantyne, B., Chapman, K., Cumminger, J., Green, E., . . . Whynot, A. (2012). Managing symptoms during cancer treatments: Evaluating the implementation of evidence-informed remote support protocols. Implementation Science, 7, 110.

    Stacey, D., Bakker, D., Green, E., Zanchetta, M., & Conlon, M. (2007). Ambulatory oncology nursing telephone services: A provincial survey. Canadian Oncology Nursing Journal, 17(4), 1–5.

    Stacey, D., Carley, M., Ballantyne, B., Skrutkowski, M., Whynot, A., & the Pan-Canadian Oncology Symptom Triage and Remote Support (COSTaRS) Team. (2014). Perceived factors influencing nurses’ use of evidence-informed protocols for remote cancer treatment-related symptom management: A mixed methods study. Retrieved from http://ejoncologynursing.com/article/S1462-3889(14)00191-4/fulltext

    Stacey, D., Carley, M., Kohli, J., Skrutkowski, M., Avery, J., Bazile, A.M., & Budz, D. (2014). Remote symptom support training programs for oncology nurses in Canada: An environmental scan. Canadian Oncology Nursing Journal, 24(2), 78–88.

    Stacey, D., Chambers, S.K., Jacobsen, M.J., & Dunn, J. (2008). Overcoming barriers to cancer-helpline professionals providing decision support for callers: An implementation study. Oncology Nursing Forum, 35, 961–969. doi:10.1188/08.ONF.961-969

    Stacey, D., Macartney, G., Carley, M., Harrison, M.B., & the Pan-Canadian Oncology Symptom Triage and Remote Support Group (COSTaRS). (2013). Development and evaluation of evidence-informed clinical nursing protocols for remote assessment, triage and support of cancer treatment-induced symptoms. Retrieved from http://hindawi.com/journals/nrp/2013/171872

    Yank, V., Laurent, D., Plant, K., & Lorig, K. (2013). Web-based self-management support training for health professionals: A pilot study. Patient Education and Counseling, 90, 29–37.

    About the Author(s)

    Dawn Stacey, RN, PhD, CON(C), is a professor in the School of Nursing at the University of Ottawa and the Ottawa Hospital Research Institute in Ontario; Myriam Skrutkowski, RN, MSc, CON(C), is a research nurse coordinator at McGill University Health Centre Research Institute, Montreal General Hospital, in Quebec; Meg Carley, BSc, is a clinical research coordinator at Ottawa Hospital Research Institute; Erin Kolari, RN, BScN, CVAA, is a primary nurse at Northeast Cancer Center in Sudbury, Ontario; Tara Shaw, RN, CON(C), is a clinic RN at Nova Scotia Cancer Centre, Capital Health, in Halifax; and Barbara Ballantyne, RN, MScN, CON(C), CHPCN(C), is an advanced practice nurse in palliative care at Northeast Cancer Center, all in Canada. This study was funded by a Knowledge to Action grant from the Canadian Institutes of Health Research. Stacey can be reached at dstacey@uottawa.ca, with copy to editor at ONFEditor@ons.org. (Submitted June 2014. Accepted for publication October 9, 2014.)

     

    References

    Baker, R., Camosso-Stefinovic, J., Gillies, C., Shaw, E.J., Cheater, F., Flottorp, S., & Robertson, N. (2010). Tailored interventions to overcome identified barriers to change: Effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews, 3, CD005470. doi:10.1002/14651858.CD005470.pub2
    Barth, J., & Lannen, P. (2011). Efficacy of communication skills training courses in oncology: A systematic review and meta-analysis. Annals of Oncology, 22, 1030–1040. doi:10.1093/annonc/mdq441
    Berkhof, M., van Rijssen, H.J., Schellart, A.J., Anema, J.R., & van der Beek, A.J. (2011). Effective training strategies for teaching communication skills to physicians: An overview of systematic reviews. Patient Education and Counseling, 84, 152–162.
    Bhanji, F., Gottesman, R., de Grave, W., Steinert, Y., & Winer, L.R. (2012). The retrospective pre-post: A practical method to evaluate learning from an educational program. Academic Emergency Medicine, 19, 189–194. doi:10.1111/j.1553-2712.2011.01270.x
    Bowen, S., & Graham, I.D. (2013). Integrated knowledge translation. In S. Straus, J. Tetroe, & I.D. Graham (Eds.), Knowledge translation in health care: Moving from evidence to practice (2nd ed., pp. 14–23). Oxford, England: Wiley-Blackwell.
    Brouwers, M.C., Kho, M.E., Browman, G.P., Burgers, J.S., Cluzeau, F., Feder, G., . . . Makarski, J. (2010). Development of the AGREE II, part 2: Assessment of validity of items and tools to support application. Canadian Medical Association Journal, 182, E472–E478.
    Brouwers, M., Stacey, D., & O’Connor, A. (2010). Knowledge creation: Synthesis, tools and products. Canadian Medical Association Journal, 182, E68–E72. doi:10.1503/cmaj.081230
    Canadian Nurses Association. (2007). Telehealth: The role of the nurse. Retrieved from http://cna-aiic.ca/~/media/cna/page-content/pdf-en/ps89_telehealth_e.pdf
    D’Eon, M., Sadownik, L., Harrison, A., & Nation, J. (2008). Using self-assessments to detect workshop success: Do they work? American Journal of Evaluation, 29, 92–98.
    Forsetlund, L., Bjørndal, A., Rashidian, A., Jamtvedt, G., O’Brien, M.A., Wolf, F., . . . Oxman, A.D. (2009). Continuing education meetings and workshops. Cochrane Database of Systematic Reviews, 2, CD003030.
    Graham, I.D., Logan, J., Harrison, M.B., Straus, S.E., Tetroe, J., Caswell, W., & Robinson, N. (2006). Lost in knowledge translation: Time for a map? Journal of Continuing Education in the Health Professions, 26, 13–24.
    Grimshaw, J.M., Eccles, M.P., Lavis, J.N., Hill, S.J., & Squires, J.E. (2012). Knowledge translation of research findings. Implementation Science, 7, 50. doi:10.1186/1748-5908-7-50
    Harrison, M.B., Graham, I.D., van den Hoek, J., Dogherty, E.J., Carley, M.E., & Angus, V. (2013). Guideline adaptation and implementation planning: A prospective observational study. Implementation Science, 8, 49. doi:10.1186/1748-5908-8-49
    Harrison, M.B., van den Hoek, J., & the Canadian Guideline Adaptation Study Group. (2012). CAN-IMPLEMENT©: Guideline adaptation and implementation planning resource. Retrieved from http://cancerview.ca/idc/groups/public/documents/webcontent/can_impleme…
    Hill, L.G., & Betz, D.L. (2005). Revisiting the retrospective pretest. American Journal of Evaluation, 26, 501–517.
    Howard, G.S. (1980). Response-shift bias: A problem in evaluating interventions with pre/post self-reports. Evaluation Review, 4, 93–106.
    Howard, G.S., Millham, J., Slaten, S., & O’Donnell, L. (1981). Influence of subject response style effects on retrospective measures. Applied Psychological Measurement, 5, 89–100. doi:10.1177/014662168100500113
    Kirkpatrick, D.L., & Kirkpatrick, J.D. (2005). Evaluating training programs: The four levels (3rd ed.). San Francisco, CA: Berrett-Koehler Publishers.
    Knowles, G., Hutchison, C., Smith, G., Philp, I.D., McCormick, K., & Preston, E. (2008). Implementation and evaluation of a pilot education programme in colorectal cancer management for nurses in Scotland. Nurse Education Today, 28, 15–23. doi: 10.1016/j.nedt.2007.02.002
    Légaré, F., Politi, M.C., Drolet, R., Desroches, S., Stacey, D., & Bekker, H. (2012). Training health professionals in shared decision-making: An international environmental scan. Patient Education and Counseling, 88, 159–169. doi:10.1016/j.pec.2012.01.002
    Macartney, G., Stacey, D., Carley, M., & Harrison, M.B. (2012). Priorities, barriers and facilitators for remote support of cancer symptoms: A survey of Canadian oncology nurses. Canadian Oncology Nursing Journal, 22(4), 235–247.
    Pasacreta, J.V., Kenefick, A.L., & McCorkle, R. (2008). Managing distress in oncology patients: Description of an innovative online educational program for nurses. Cancer Nursing, 31, 485–490.
    Perry, P. (2011). Concept analysis: Confidence/self-confidence. Nursing Forum, 46, 218–230. doi:10.1111/j.1744-6198.2011.00230.x
    Stacey, D., Bakker, D., Ballantyne, B., Chapman, K., Cumminger, J., Green, E., . . . Whynot, A. (2012). Managing symptoms during cancer treatments: Evaluating the implementation of evidence-informed remote support protocols. Implementation Science, 7, 110.
    Stacey, D., Bakker, D., Green, E., Zanchetta, M., & Conlon, M. (2007). Ambulatory oncology nursing telephone services: A provincial survey. Canadian Oncology Nursing Journal, 17(4), 1–5.
    Stacey, D., Carley, M., Ballantyne, B., Skrutkowski, M., Whynot, A., & the Pan-Canadian Oncology Symptom Triage and Remote Support (COSTaRS) Team. (2014). Perceived factors influencing nurses’ use of evidence-informed protocols for remote cancer treatment-related symptom management: A mixed methods study. Retrieved from http://ejoncologynursing.com/article/S1462-3889(14)00191-4/fulltext
    Stacey, D., Carley, M., Kohli, J., Skrutkowski, M., Avery, J., Bazile, A.M., & Budz, D. (2014). Remote symptom support training programs for oncology nurses in Canada: An environmental scan. Canadian Oncology Nursing Journal, 24(2), 78–88.
    Stacey, D., Chambers, S.K., Jacobsen, M.J., & Dunn, J. (2008). Overcoming barriers to cancer-helpline professionals providing decision support for callers: An implementation study. Oncology Nursing Forum, 35, 961–969. doi:10.1188/08.ONF.961-969
    Stacey, D., Macartney, G., Carley, M., Harrison, M.B., & the Pan-Canadian Oncology Symptom Triage and Remote Support Group (COSTaRS). (2013). Development and evaluation of evidence-informed clinical nursing protocols for remote assessment, triage and support of cancer treatment-induced symptoms. Retrieved from http://hindawi.com/journals/nrp/2013/171872
    Yank, V., Laurent, D., Plant, K., & Lorig, K. (2013). Web-based self-management support training for health professionals: A pilot study. Patient Education and Counseling, 90, 29–37.