BM Thinking Out Loud

ADLT 602

            This class experience has been very appropriate for my current stage of development in understanding organizational development and learning, as well as applicable to my current position at work. Caffarella’s (2002) interactive model for program development is such an excellent tool which I know I will be able to use in my work. I really enjoyed working through the program development process; using the tools and skills I learned in this course. The model covers all the areas that program development might entail and is flexible enough to fit any application.

            I think the elements of the model that were most helpful to me included determining the context of the environment for which the program is to be implemented, devising transfer of learning plans, and formulating evaluation plans. In the past, I have given some consideration to whether the program and its structure and content would be appropriate to the group for which it was intended. However, now I am completely sold on the process described by Caffarella’s (2002) for discovering the context of the organization and the larger environment in which it operates by examining the people and artifacts that exist within. It only makes since that a developer would want to know the culture, people, and circumstance for which the program is being planned.

            Planning for the transfer of learning is so important to the success and overall effectiveness of any training program. Having been an elementary level school teacher, this area has been the focus of many of my educational endeavors. Getting the message across is one thing, but making it stick by ensuring its meaningfulness to the learners is quite another feat. Working with adults presents a new set of challenges in this area. I like the way Caffarella’s (2002) explains the very simple factors that can influence the ability and likelihood that learners will apply what is learned in training. I think that the success of enabling the transfer of learning is in the deliberate forethought and planning put into discovering ways to address both the things that might prohibit retention and application and those things that will enhance them. I also like the idea that this process should include planning for transfer of learning before, during, and after the learning event.  

             At work, “in the real world”, very little thought or effort is put into evaluating the effectiveness of the programs we develop. Once the decision to create and deliver the program has been made, there is not usually much interested in demonstrating the usefulness of the program or its perceived value to the participants. The most we usually do is the level one reaction survey. I am glad that this course reinforced the importance of considering the both summative and formative evaluation of learning programs throughout their development, implementation, and following the completion of the course. Even if the company is not particularly interested in measuring the effectiveness of programs, having an appropriate structure for judging the value or worth of learning programs will benefit not only the participants and the developers, but the process of evaluating will improve future programs, as well as sharpen my own skills as a program developer.  

            Having the privilege of learning about the experiences of my classmates was one of the best parts of this course. Sharing the harsh realities of the workplace, the humorous and sometimes odd experiences of everyone in class really made the content come to life. I have to say that I really enjoy learning with everyone in the class. The use of blogs and being able to reflect on what we were learning was another thing I really enjoyed about the course. While we did not do much responding to each other’s blogs, I believe that reading about the reflections of our classmates did much to reinforce the course content and bring us together in our learning experience. Learning from each other is perhaps one of the best means of making learning real and lasting.

            This has been a great course and I wish I had taken it a long time ago. I will highly recommend this course to my coworkers!    Buddy   

As I have been doing interviews for my cultural analysis, I am struck by how similarly most people feel about the questions I am asking. I thought that I would get a mix of answers and opinions, as I am talking to a variety of people from many different departments. However, I am finding that people generally have the same interpretations of events and behaviors here on site. The most difficult part of this assignment for me is trying to identify the underlying assumptions upon which these opinions are based. During the interviews we try to get to identify these assumptions, but generally people have not ever thought about that before and the concept is new to them.
One interesting thing I have noticed throughout these interviews is that many people seem to feel like victims at the mercy of the leadership team and its lack of leadership over the years, as well as several ‘famous’ acts of favoritism and vengeance over the years. I think the strong hierarchy in this company has somehow caused the feeling of being somewhat powerless and of being victimized by authority. This assignment is really an interesting one, however I am not sure I am really knowledgeable enough to really get at the heart of the basic assumptions that influence the behavior at this company.

I found this week’s readings on reporting evaluation activities and findings very interesting. This is because much of what is possible does not happen in my organization, at least not in my department. The most likely audience to receive a report on evaluation findings would be our site leadership team. The first rule about reporting to them is that no one is worthy of more than 20 min. of their time, and this includes any time needed to answer questions. Reports must be concise, to the point and contain only info. that they are interested in seeing. There are rules around the format and length of presentations and there are even company ‘logo police’ who monitor what that team is exposed to. I have never seen a ‘working session’ to prepare for such reports in my department. Usually, the presenter is required to submit their proposal to the dept. head for approval. With the exception of the executive report and brief media presentations, many of the other options presented in the text would be frowned upon in our company. I must say that it is possible to submit comprehensive reports to the leadership team, but it has been my experience that the chances of their reading it are slim to none. I know we have a lot of room for improvement here.  Buddy

I enjoyed reading chapter 14 which is about organizational ‘midlife’. I believe that is where my site is now. Our parent company is very mature, but our site has only existed for 30 years. Schein points out that leaders at this stage are less able to bring about cultural changes through directives and by imposing their own assumptions on the company. By midlife companies have established cultures and subcultures that do not respond easily to these things. In addition, the larger our company grows the less influence the leader has on his own. The layers of management are growing and the distance between employees and leadership grows in proportion to that, at least it has at my company.
In this chapter Schein describes the ways companies differentiate as they grow. If I consider just my own site, we are differentiating functionally and hierarchically. The departments in our organization are getting larger and moving to systems that make their work more efficient. The issues we deal with as a whole site are often surrounding getting all those systems to work together and training people on the many systems. This has served to separate people in different departments. Each department develops its own subculture. Members of different departments come up with their own ways of functioning and their own vocabulary and their own inside opinions of other departments. The challenge for leadership is to ensure that all these growing subcultures remain connected with the larger company culture and don’t split off to the point that they are unable to function effectively within the organization. This is what is happening at my company to some extent. I see the leadership struggling with this and trying things like
‘transformational change’ and moving to ‘top talent’ (their version of high performers), and looking at who really makes money for the company and differentiating their compensation and benefits to reflect their importance to the company – and subsequently creating more divisions than existed before.
I know I should trust that the leadership knows what they are doing and they really do have the best interest of the company in mind. However, at this point I am not seeing much evidence of that. I see that the culture is suffering and subcultures are being strengthened as people commiserate and try to defend or insulate themselves. There is a lot of talk about the culture of the company in HR, but not a lot of real influence; at least not yet.

     This week we read about individual and focus group interviews. I had the opportunity to conduct a focus group at work about a month ago. I did have the help of someone experienced at this process and was able to learn a great deal from the experience. The focus group consisted of 12 people managers from various departments in the organization where I work. The topic was performance management at our site and I asked five key questions about their experiences with the existing process. The managers were more than happy to participate and shared a great deal about what they liked and didn’t like about the current process. Both I and my co-facilitator took notes.

 I did do many of the things talked about in chapter 10 of Ruff-Eft & Preskill. I chose a cross functional group with enough experience with the current performance management process to be able to contribute to the conversation. I invited them all to participate in the focus group in an email which explained the purpose and design of the group, however I did not give them the questions before the meeting. I used a semistructured approach and kept to the five questions I had prepared. I also kept the response time to ten minutes per question with ten minutes for any final thoughts. I did not want to keep the group more than an hour. I am not sure that I was always as neutral in my responses and facial expressions as the text suggests. This was partially due to both my familiarity with the group and the subject at hand.

     The reason for this focus group was more than just taking the pulse of the process. I am in the process of developing some just-in-time training for the managers (and separate training for all non-people managers) and I wanted to gather information on their experiences with the process and suggestions for any improvements they would like to see. I have followed up on any questions raised by the group and am working to give the group some feedback on those issues. I will also remind the group that we are going to reconvene in the spring to talk about the upcoming training and their thoughts on its usefulness (both for them and for their direct reports).

     I was happy to see how much participation took place in that focus group and the willingness of the participants to share their experiences, both good and bad. I know I have much to learn yet, but I really did enjoy the experience and look forward to the next focus group.

I wonder what the shelf life is of the culture that founding leaders impart to a developing organization. I ask that question because as I read chapters 12 & 13 in Schein (which focus on how leaders begin culture and how leaders embed and transmit culture) I realize that my own organization (globally) was founded 125 years ago. I suspect that there have been many leaders along the way with different ideas of what the culture should be. The site at which I work in the US is 30 years old. Not quite so mature, yet it has been through a number of leaders and leadership teams. I am sure much of the founding ideals exist in some form today, yet I wonder how much. In the past two years we have gone through three site presidents, each very different form the other. I am really looking forward to the cultural analysis that I am embarking upon. I have often wondered how the espoused values got to be so far removed from the behaviors of the leadership. There is a great list of mechanisms through which leaders embed values and assumptions on page 246 of Schein. I am hope to be able to get a look at how those mechanisms are used in my organization and with what results. This should prove to be very interesting indeed as we are going through ‘transformational change’ at work and we have just gotten a new president. I just hope that being such a novice at this does not skew the findings a great deal.

I am trying to wrap my mind around the various ways to evaluate my program. I think I am leaning towards a combination of using information from the needs analysis, the program objectives, and the transfer of learning strategies. I want to use a level one (reaction) and a level 3 (behavioral) evaluation at the end of the program. I would like to gather information on the content and delivery, as well as the actual ability of the learners to implement the skills taught throughout the program. I would also like to gather information regarding the performance feedback received and the quality of performance discussions through an employee survey to be administered approximately three to six months following the program’s completion. The information gathered will be used for both formative and summative purposes. 

I will get the level 1 evaluation information through the use of a reaction survey administered at the end of the program in the last training session. I believe I will obtain the level 3 evaluation information through auditing the work of the participants over time, following the program. The leadership team of the company will be doing the auditing using a ratings tool that I am developing. The information gathered from thier audits should indicate the success of the program.

I believe I will interpret the data obtained from the evaluation methods both qualitatively and quantitatively. I will want to provide hard data for the leadership team, related to the number of performance plans completed correctly and on time, as well as the types of responses I receive on the employee surveys given following the program.

In truth, I am not at all sure how I will tabulate and communicate all this data at this point. But I have the feeling I am going to figure that out very soon.

Field Interview -

                I interviewed Dr. Debrah Westin who works with the Chesterfield County Adult Education Program as an instructor and program planner for teaching English to Speakers of Other Languages (ESOL).  Dr. Westin works with literacy, writing, listening and conversation classes, as well as higher level conversational classes. Her learners are from various ethnic backgrounds including Hispanic, Japanese, Chinese, and others. She also works as a private tutor for adults for ALSTOM an international power company.

                In answer to my question about any special considerations that she addresses with her learners, Dr. Weston told me that she takes into account the whole person when planning for her adult students. She described her learners as people generally in their 20’s who work long hours, and who are often working two jobs to make ends meet. Most of her learners have children and families to support. In addition a great number of them live in homes consisting of many family members. As her students often come to class very tired, Dr. Westin always builds in physical  and laughter activities in the curriculum in order to keep them active and alert in class.

                Another consideration that Dr. Westin works to address is the fact that in many of the cultures from which her students come the teacher is revered “next to God”, as she put it and she has to work to overcome the separation that often exists between the learners and the instructor. She likes to build in activities which will build a working relationship between her and the learners.

                Dr. Westin told me that the greatest constraint she faces when planning for this program is that she must show gains in the learner’s as a result of the programs. She is required to conduct pretests and posttests to show these gains. Dr. Westin feels this testing is both an asset and a liability. While the assessments can show growth, they are not always the best way to measure this progress. She finds that some instructors are tempted to do things that cause these assessments to show positive results which speaks to the ethics and motivation of some instructors.

                Dr. Westin also mentioned that the fact that the learners come into the programs at different ability levels can be a constraint as well.  In fact, many of her learners are not literate in their first language. This causes problems in the teaching of grammar and writing skills as the students don’t have a familiar reference to draw upon. Those who may be ahead of others can become impatient or board when the content is paced to accommodate the slower learners. In order to compensate for this Dr. Westin is sure to build in special projects for the more advanced to challenge them and keep them engaged. In her higher level classes she has such students do things like compare advertisements from other cultures with American advertisements and talk about differences in approach and culture. In this way she is teaching culture as well as literacy.

                I asked Dr. Westin where she gets ideas for programs. She told me that she goes to several key resources for ideas. These include the Adult Learning Resource Center at VCU, an internet site entitled Dave’s ESL Café’, and other ESL instructors to both adults and children. She also consults with a psychologist at VCU who works with international students.  There is a commercial curriculum that Chesterfield County uses with the adult ESOL students called ‘Stand Out’ which offers texts and lesson units. Dr. Westin told me that the county uses observations to ensure that the required curriculum topics are covered.

                A challenge that Dr. Westin has faced beyond those mentioned earlier is that some students are unwilling to speak out in class. This is due to a variety of reasons such as cultural norms, being shy about their lack of knowledge of English and personality. One way Dr. Westin works to overcome this challenge is by adding humor into the class. She will tell them that she can’t hear them in a joking way. She also tries to give them practical examples of why speaking out can be important, as in example that they may be speaking to someone about a job and the loudest person will get the job.

                Dr. Westin shared with me that one of the most successful programs she has planned has been one on giving directions and learning one’s neighborhood. In this program students learned how to give and receive directions to local landmarks such as Wal-Mart. In this program she included experiences using conversational English and writing to accomplish the objectives.  Dr. Westin also included physical activity, songs and leaning that involved the use of the whole body, as well as the mind. She pointed out that the use of many different interest building activities made the program successful.

                Dr. Westin spoke of adult learning theory as guiding principles in her program planning, but I was surprised that teaching adult ESOL learners requires the incorporation of so many attention getting and interest holding activities. I did not know that the learner’s need to work multiple jobs and their life responsibilities impacted their ability to pay attention and participate in learning activities to such a great extent. I was also interested to learn that Dr. Westin struggled with some of the same issues I had to deal with as an elementary school teacher. In particular, some students with advanced abilities became bored if they were not continually challenged.  Dr. Westin pointed out that some of the advanced students sometimes became impatient and would interrupt other students and correct their grammar. I would not have expected that among adult learners.

                While there was mention of some of the elements of the interactive model of program planning that we have learned about in class, it did not seem that Dr. Westin’s approach was as detailed. This may be in part to the demographic of her learners.  The elements that she talked about most were discerning the context, designing instructional plans, devising transfer of learning plans, and sorting and prioritizing program ideas. She did not mention having to build support for the programs, probably because they are already supported by the county and funded by the federal and state governments.  It did not seem that Dr. Westin had to formulate evaluation plans other than the pretests and posttests as she has only to demonstrate growth in her learners to keep the funding. I learned from our conversation that many of the objectives were focused around conversation, listening, reading and writing English and that her work involved planning sub-objectives and strategies designed to enhance these skills. I gathered that conversational English skills take priority in her planning as that was the greatest need of the majority of her learners.

                Perhaps the most important thing I learned from this interview is that there can be quite a variety of approaches to program planning. It is clear that the target audience has as big an impact on the planning process as the content. Due to the population that is being served in this example, the content and means of instruction take priority over many of the other elements of program design. I also learned a little more about the influence that the stakeholders can have in the design and implementation of such programs. I did have some idea of this already having had the experience of working in the public school system and having to teach Standards of Learning objectives to satisfy the requirements of the state. One other thing I learned about program planning from this interview is that meeting the needs of a given adult population can go far beyond providing the content of the curriculum in an organized way. Providing the physical, emotional and social needs of the learners can have a far greater impact than can be measured in any program evaluation. This leads me to think that discerning the context may just be the most important aspect of program planning.

          I enjoyed talking with Dr. Westin and learning about the program planning that she does in her work with ESOL learners. It is very helpful to learn about examples of program planning that take place in the context of different environments and with different populations.  I believe that thinking through the experiences of other program planners has been one of the most effective learning strategies for me as I work to understand and apply these skills.

     I found the content of chapter 7 about the deeper dimensions around which shared basic underlying assumptions form struck a nerve with me. It is so true that people don’t often share a basic understanding of what these things mean, especially around the nature of human nature and truth. I find that I often have to try to decipher what other people must believe about these things in order to make sense of their actions and words.

     I know that my understanding about human nature and truth is not popularly shared, but to me it is nonetheless valid. However, I often run into people with very different beliefs who assume that their thinking is the only correct thinking and who exhibit no tolerance for any other view.

            Even within the framework of the organization and what the cultural beliefs appear to be, I find intolerance and assumption run rampant. There have been times when the difference between what I understand and accept and what others or even the organizational culture will accept are very different things. I know that in order to discover what is accepted as truth in an organization takes lots of time and interactions with its members, which is why I am always so cautious when the leadership comes out with a new initiative to ‘transform’ the culture. How can we ‘transform’ what so many people within the organization don’t really understand in the first place?

In reading through the text about program evaluation I am reminded of my research methods and educational evaluation classes. I do like the format for planning out the evaluation proposal. It helps me clarify what it is I am interested in learning from the program. It is helpful to think about the stakeholders and what they would like to see come of the program, as well as to think about the key questions that I hope will be answered. I added a small number of behavioral questions for myself (see page 135 in Russ-EFT, Preskill 2001)that I think should be answered in relation to the different behaviors that I would like to see take place in the program participants. I also think it is helpful that the purpose statement includes not just the overall desired outcome of the program, but information about how the results will be used. This gives the program more of a purpose than just to educate or transmit information.  As I mentioned in the class wiki, I think I will use the one group pre/posttest design in my program evaluation. This will meet the needs of my organization, as well as provide me the means to check for transfer of learning over time and in practice. I hope things work out as well I think they will. Oh by the way, I received a directive today from the executive director of my department that she wants my program to be delivered to the entire site. This will prove to be quite an undertaking considering how many people are on site and the great amount of other training that everyone is involved in at this time of the year. I hope I don’t end up more of a nuisance than a help.