Taking open answers seriously: gaining insights into the perilous undercurrent of organizational change

As Clarke (2013, p. 31) phrased powerfully, a “sometimes perilous undercurrent” is common in many organizational change projects. A certain proportion of those undergoing the change tends to have reservations about proposed changes. Their responses may vary from constructive criticisms to open hostility (e.g. McDermott et al., 2013; Oreg et al., 2011). Especially covert perceptions and reactions may be critical (Boiral, 2003; Piderit, 2000). Thus, gaining insights into its undercurrent may be essential for change projects to succeed, yet is at the same time out of its very nature difficult to achieve. Change recipients will often refrain from openly voicing their reservations, yet obstruct the change through their actions (Ybema and Horvers, 2017). It follows that it is at the same time essential as well as inherently difficult to shed light on the “perilous undercurrent.”

To evaluate and guide change projects, surveys are often used to gain insights into change recipients' opinions (O'Cathain and Thomas, 2004). Surveys are designed to find statistically generalizable findings. Yet surveys largely or exclusively consist of pre-conceived questions related to the specific change effort concerned which cannot tap into unforeseen reactions (Evans et al., 2005). Open questions of the type “Do you have additional remarks?” offer respondents to voice their concerns beyond the issues envisaged by the survey’s designers. As such, they may be suitable for gaining a better view below the surface and advancing our understanding of the undercurrent of organizational change.

In this paper, we explore whether and how analyzing free-text comments as answers to an open survey question may enhance our view of the undercurrent and unforeseen responses to an organizational change project. Our data consist of answers to an open question of a survey evaluating the implementation of “lean” and Lean-Led Hospital Design (LLHD) in a Dutch hospital (Grunden and Hagood, 2012; Schouten et al., 2021; Chbaly, 2022). Implementing lean alone can have substantial consequences for the staff involved, which are likely to be amplified when lean principles are solidified in the design of a new hospital building.

During the evaluation of the implementation of change in a Dutch hospital, we found that, beyond the results from the survey, the free comments contained a broad variety of staff responses indicating an undercurrent. Analyzing these comments, we identified three different categories of recipients' responses to the implementation of a lean facility design: dismissive – lean unrelated, dismissive – lean related and supportive – lean related.

Based on these findings, we contribute to the existing body of knowledge in two main ways. First, we show that “free comments” allow gaining further insights into the nature and variety of (hidden) responses as a manifestation of the undercurrent in organizational change, including, and most notably, negative responses unrelated to the concept under study or the changes associated with it. Second, we revealed employee characteristics that can be associated with particular responses, allowing change managers to capitalize on constructive comments of “critical friends.”

Surveys are commonly used to collect, code and analyze employees' responses to change projects (cf. Borg and Zuell, 2012). They may contain, generally at the end, an open question to invite respondents' to give comments of their choice (Garcia et al., 2004). Such “Anything else?” items (Borg and Zuell, 2012, p. 208) might deliver deeper insights into how a project’s change is actually experienced (cf. Rich et al., 2013). As Ellonen et al. (2018, p. 4) put it: “free-text comments may give important further information and offer a context for other responses, as well as bring out issues that quantitative survey data alone cannot reveal”. As such, free-text comments might have the potential to enhance our view of employees' responses to change, and as such, can be beneficial for practitioners, like change agents and managers, in a specific organizational context (Poncheri et al., 2008).

Whilst analyzing such free-text comments has been recommended for almost 30 years, they are still used rarely, and remain remarkably absent in studies of organizational change in particular (Riiskjær et al., 2012; Poncheri et al., 2008; Ellonen et al., 2018). Garcia et al. (2004) searched many well-known textbooks on survey methods and qualitative research and did not find a single reference for their use. We conducted a literature search using Garcia et al. (2004) as our starting point, as this was the first paper about the use of these “anything else” questions after the year 2000. Tracking Garcia et al. (2004) through Google Scholar resulted in 141 references (searched 10 April 2022). Their recommendation to make better use of free-text comments has been primarily followed in medical disciplines, particularly to gain insights into patient experiences (Rich et al., 2013). Only three studies concern an organizational setting (Borg and Zuell, 2012; Evans et al., 2005; Poncheri et al., 2008) or addressed using these comments methodologically (Ellonen et al., 2018; Marcinowicz et al., 2007; Rich et al., 2013).

There may be several reasons why free-text comments are hardly used in general and in studies of organizational change in particular: (1) a perception that analyzing them can be time-consuming (Borg and Zuell, 2012), (2) a debate whether they should be considered qualitative or quantitative data (O'Cathain and Thomas, 2004), and (3) the assumption of a potential negativity bias (Poncheri et al., 2008). Indeed, Poncheri et al. (2008) warn that the content of free comments can be “disproportionately negative”, but they also stress that these comments can be useful to practitioners as they “may alert potential problems in a particular department” (p. 626). This raises the question of whether the respondents have particular characteristics. In the studies of Poncheri et al. (2008) and Borg and Zuell (2012), this was, however, not the case.

Drawing on these insights, we seek to empirically explore hospital staff’s responses to change initiatives.

The free comments at the end of a questionnaire evaluating hospital staff’s responses about a new hospital facility based on LLHD were used. The survey of 26 items was held among hospital staff 18 months after moving into the new lean-designed facility (June 2018). The questionnaire was sent by email to 1,613 employees of the hospital by the first author. The answers to the final item, “If you have any additional comments, you can enter them here”, are used here as our primary data source. In total, 360 employees responded by completing the survey till the last question. This resulted in a response rate of 22.3% (n = 360). In line with hospital staffing, our largest response group consists of nurses (34%; n = 121).

The 360 respondents left 177 free-text comments. Those containing a “good luck” note or remarks on the questionnaire itself were excluded. This resulted in a primary data set containing 152 free comments.

Further, several recipients' characteristics were identified that could help to gain better insights into the respondent’s response to change (Bowen, 2006) such as their tenure at the hospital, participation in lean training, whether they felt informed about the lean concept and the experienced support of management and medical board. For comparison purposes, the closed question on recipients' attitude to lean was included as well. In total, 11 out of the total 26 items were included in this study (including the last open question). Detailed data are available from the first author.

The answers to the included closed questions about recipients' characteristics of the “free comment respondents” (n = 152) were compared with the answers of respondents who did not leave a free comment (n = 183) to check for a possible bias among the respondents (Poncheri et al., 2008). To check for over- or under-representation, we focused on the larger respondent groups (n > 10) with a difference of more than 10%, as findings based on small numbers are more prone to coincidence. The only difference that was found is that employees with shorter tenure left an open comment less often than employees with longer tenure (>25 years). Overall, however, no significant differences were found between both groups.

An inductive analysis was used, aimed at finding themes in the data that capture the essence of the experience or peoples' responses to varied situations or contexts (Bowen, 2006). The qualitative data analysis consisted of three main stages of coding – open, axial and selective (Bowen, 2006). In the open coding, each comment was allocated to one or more topics it addressed. This resulted in an extensive list of topics, asking for a more general construct to get organized. In the second coding stage, two axial dimensions emerged from the analysis: (a) whether the comment was either mainly positive or negative and (b) whether it was mainly about the concept (lean or LLHD) or another, not lean-related issue (such as working culture, interior design or the new waste system). This resulted in the designation of each free comment to one of four categories. We used a matrix to display the results of this step to visualize reduced data systematically and understandably (Miles and Huberman, 1994).

Finally, we looked further into the narratives of these open comments and identified different elements constructing the responses within each quadrant. In this analysis, we used the functions and consequences of the usage of specific language to understand the meaning of the respondents' comments as a means to develop categories. Each comment was evaluated based on the assumed underlying structures of coherence (Potter et al., 1990). To do this, all written comments were coded and categorized in their original language (Dutch) by the first author (van Nes et al., 2010). When writing the paper, the selected comments were translated by the first author. Given that these comments have a predominantly factual character, the authors felt that they well-presented the intended expression of the respondents.

A framework along two dimensions was constructed based on the data (see Figure 1). As such, three categories of responses towards change were identified: (1) dismissive – lean unrelated, (2) dismissive – lean related and (3) supportive – lean related. Within each category different subcategories of responses emerged, delivering more detailed insights into the nuances of different responses. The labels of these subcategories refer to the circles in Figure 1. In this analysis, a fourth category (positive, lean unrelated) was excluded from further analysis, because this only involved one respondent.

The first and largest category of responses is dismissive and not related to the change project and concept as such (lean or LLHD). Within this category, we found four analytically distinct responses (subcategories A, B, C and D). In the largest subcategory (A; n = 41), respondents gave a “single-issue response”: the new facility is OK but one specific issue is malfunctioning. Their responses relate to aspects that are linked to the fact that a new hospital facility, like a new home, always brings (a lot of) new routines, concepts, innovations and change. These respondents are disappointed in the specific result of one of these parallel innovations/changes and the fact that this issue should soon be solved or appropriately addressed. In general, they made rather short comments in a business-like tone of voice.

The second subcategory of responses (B; n = 20) contains references to structural issues respondents find annoying. These respondents expressed a systemic response illustrating the new facility as a whole is not working. They enlisted many failing innovations/changes implemented parallel to LLHD/lean in randomized order and in a business-like tone of voice, although most comments have a slightly sarcastic or emotional undertone.

Respondents in the third subcategory (C; n = 25) used the last open question as an opportunity to bring non-related contextual issues to attention. These issues are representative of issues in Dutch healthcare in general such as labor market problems creating a lack of nursing staff, accompanying high working load and absenteeism. Respondents in this subcategory tended to use stronger/emotional expressions to emphasize their points.

Finally, three comments concern expressions of nostalgia (subcategory D; n = 3). These respondents expressed their longing for the old – familiar – hospital facility. Although the relatively small number of comments in this subcategory is limited to this one issue, this nostalgic undertone can also be found in some of the comments in other subgroups.

In this category, respondents expressed struggles with their disappointment over the results of working in the new building and blame it on the concept under study (lean/LLHD). In this group, four analytically distinct subcategories of responses emerged. The largest subcategory (E; n = 26) contains comments in which respondents coupled lean with not related negative outcomes or feelings. This signals that they used the concept as a more general label for all causes that, in their opinion, result in things going wrong in the new facility.

The second subgroup (F; n = 8) contains comments in which respondents associated the lack of a sense of realism within the LLHD implementation team with the resulting negative outcomes in the new facility. These respondents felt that insufficient input was requested from hospital staff or that the project team did not listen to them. These comments often have a blaming undertone and/or an emotional undertone of not being heard.

The respondents in the third subcategory (G; n = 9) appeared to consider lean as a concept with a failed or even absent implementation process, leading to negative outcomes. They addressed a lack of support during implementation.

The last subcategory of responses (H; n = 3) represents a generally neutral attitude towards the concept itself, but these respondents indicated they feel that the concept is not applicable in their working environment.

This smallest category of free comments refers to lean and/or LLHD in a positive tone. These are comments of respondents who seem to have embraced the concept, although most of them expressed different issues regarding the implementation process. In this group, two subcategories can be identified. The first subcategory of respondents (I; n = 15) stressed they “believe” in the concept and they actively de-coupled (the negative results of) the implementation process from the concept. Their comments refer to certain conditions that need to be improved for successful implementation.

The last comment (subcategory J; n = 1) is from a respondent who stressed that lean was implemented successfully at the new hospital facility.

Although an ex ante analysis for respondent bias showed no difference between the respondents leaving a free comment and those who did not (see methodology), we further analyzed the specific characteristics of the respondents in the three categories (see Table 1).

Taken together, these results suggest how change recipients' characteristics relate to the three different categories of responses to change. The first group, recipients with lean unrelated dismissive responses, is typically more concerned about other innovations or change initiatives that were implemented simultaneously. A majority of this group is trained and informed about the concept (lean/LLHD), and their attitudes expressed in the closed question about lean are equally distributed amongst positive, negative and neutral. These recipients experience strong support from the board of directors and medical staff for the concept. In the second group, recipients with dismissive – lean-related responses are generally least trained in lean and least actively involved in the implementation of the concept. They experienced a strongly formalized implementation climate. This likely explains why 24% of these respondents give a positive answer to the closed question about their attitude to lean, despite their negative open comments. The third group of supportive – lean-related recipients is almost fully trained and informed about the concept. They experience the least formal support from the board of directors or medical staff for the concept.

This study makes two contributions. First, the authors show how open comments may enhance insights into the nature of the “often perilous undercurrent” of an organizational change project. Whilst the responses were quite varied, the largest share was predominantly dismissive and not related to the change initiative under evaluation. This large proportion of dismissive responses was also found in earlier research (Borg and Zuell, 2012).

This non-lean related dismissive undercurrent seemed mostly associated with other initiatives that were implemented simultaneously or with other external developments like labor market pressures (cf. Bartunek et al., 2006). Moving into a new facility result in physical, social and/or psychological disturbances and should be seen as a major change (Matthiesen and Tissington, 2008). It requires recipients to adjust their daily work routines because the change affects their working environment and their processes. Such changes are especially significant in a highly institutionalized context such as hospitals where taken-for-granted norms, values and beliefs are rather fixed (Currie et al., 2012). As shown in our findings, changes influencing hospital staffs' routines evoke (negative) feelings of loss (Conroy and O'Leary-Kelly, 2014; Feldman and Pentland, 2003). Multiple initiatives inducing change at the same time likely lead to cumulative effects (Kiefer, 2005). Where evaluative surveys aim to single out a change program’s effects in their splendid isolation, the free-text comments make clear that many change recipients themselves do not isolate their opinions from the broader organizational context. Whilst one may counteract that the projection of unrelated concerns to the change project is not justified, one has to take into account the classic “Thomas Theorem”: “if men define situations as real, they are real in their consequences” (Thomas and Thomas, 1928, p. 572; Merton, 1995). This means that whether these dismissive responses about change are real or not by origin, they are real for change recipients and may thus inform their actions.

Their second contribution involves providing more insight into recipients' characteristics that can be associated with the change initiatives' undercurrent. Especially staff with long tenure seem to express dismissive lean-unrelated responses, thereby signaling broader involvement beyond one particular organizational change (Oreg et al., 2011). Also, change recipients who are less actively involved in a change process tend to show more dismissive responses (Oreg et al., 2011). A more supportive response could be associated with more active participation in training and involvement in the design process as well as being more informed about the implemented concept (see Table 1). This is consistent with other studies assessing the impact of participation (Bartunek et al., 2006) and knowledge (Melas et al., 2012) on responses to change. Seemingly at odds with this finding is that those best informed and most supportive are also critical. These respondents can be seen as “critical friends” (Storey and Richard, 2015), staff members with critical voices that are likely to support the obtained organizational change. These “critical friends” should be fostered as they can positively impact the change process: they support the change project and simultaneously initiate continuous improvement of the implemented changes. Precisely their involvement leads them to being critical.

As all empirical studies, this has its limitations. First, it is not known how representative the respondents' survey feedback is for the hospital population at large that holds in extenso for the free comments: although their authors appear a typical representation of the respondents, there is no view on those not participating in the survey. A “pro-involvement bias” can only be assumed: those most involved seem more likely to take the effort of responding and represent the more vocative part of the hospital population. Second, change recipients' comments do not necessarily say something about their actual behavior (Ybema and Horvers, 2017). Studying the relationship between staff’s opinions or comments about organizational change and their actual behavior is recommended for further research. This will further deepen our understanding of the impact of the undercurrent and assess how perilous it may actually be. As a first step, the authors showed how open comments gain insights into the nature and variety of (lean-related and unrelated) responses and the recipient characteristics that can be associated with particular responses.

The implications for change management seem evident: free comments voice change recipients' responses, and can thus be fruitfully used to gain a deeper understanding of the reception of changes than answers to closed questions allow. Being anonymous, these comments can serve to overcome barriers to expressing one’s responses, which is often problematic in hierarchies (Tucker and Edmondson, 2003). It is particularly relevant to gain insights into the undercurrent of organizational change because they may be at the basis of interventions to curb these interventions and thus prevent them from becoming a self-fulfilling prophecy. By identifying not only different responses to the implemented change but also different categories of recipients responding in a certain way, managers can address change recipients more appropriately in a change program or project. Recipients who are well-informed and trained, tend to be less dismissive but critical. These “critical friends” can be fostered by managers, especially middle managers – often an untapped resource in terms of mobilizing organizational change – to improve the results of change processes such as lean implementations.

The authors would like to thank the hospital under study for their help in advancing insight into change recipients' responses to change and for allowing the use of the survey data including the free-text comments. The authors also highly appreciate the two anonymous reviewers for their most valuable comments.