E-Learning Corgi focuses on distance training and education, from instructional design to e-learning and mobile solutions, and pays attention to psychological, social, and cultural factors. The edublog emphasizes real-world e-learning issues and appropriate uses of emerging technologies. Susan Smith Nash is the Corgi's assistant.
Monday, August 30, 2004
Sunday, August 29, 2004
E-Learning Blogs This Month: Part 2
contributed by Susan Smith Nash
Four more e-learning blogs profiled, ranging from the chatty and personable Michael Feldstein's e-literate to Jane Knight's e-program lifesaver, the e-learningcentre, to overwhelmingly encyclopedic e-learningguru and elearnopedia. All are outstanding and a huge service to the community. They encourage the sharing of knowledge and collaborations. Now, if I could only keep my "marketing brain-flashes" to myself (!). Oh well.
elearnopedia -- http://www.elearnopedia.com/Truly outstanding resources are available here. The articles on effective online instruction are both well-researched and accessible. The articles are neatly arranged by category: Academic Researchers, Educators, Developers, General Interest, and Content Managers. The articles are easy to access and useful. For example, in "Educators," I found Marc Miller, Kelly Rainer, and Ken Corley's "Predictors of Engagement and Participation in an Online Course" to be very useful, particularly from a behaviorist perspective, with a view to self-efficacy, motivation, and perceived relevancy of course materials. It did not measure the perceived difficulty of access or comfort level with modes of content delivery, which, although it would have complicated the picture, would have incorporated a realistic factor. I would be interested in seeing how entertaining, aesthetically pleasing, or otherwise semiotically compelling icons and other visual elements promote engagement. A major flaw: The citation does not appear in the portal, nor does the URL to the direct link. Marketing Brain-Flash: Slug-fest it! Add slugs (citations, URLs, author names).
e-learningguru -- http://www.e-learningguru.com/dashboard.asp This is an amazingly content-rich site, with numerous pages and nicely organized sets of links. In addition to links to articles, weblogs, and white papers, there is a useful glossary. There is a bit of information overload, and I would enjoy seeing how our wise elearning guru (Kevin Kruse) would rate the articles - perhaps not stars, or otherwise pedestrian qualitative evaluations, but by yoga poses to indicate how it makes the reader stretch. Kevin Kruse's "Blog Roll" features unique and often-overlooked weblogs dealing with all aspects of elearning. Now, what are Kevin's thoughts and insights? I, for one, would love to read a daily op-ed piece or analysis. Marketing Brain-Flash: Kevin, tell us what you're thinking!
e-literate - http://mfeldstein.com/ Michael Feldstein's e-literate has everything I love in a blog - a sense of the person behind the blog, an engaging tone, great links, articles about all sorts of things, articulate commentary, and lively ways of classifying and thinking about e-cognition, e-learning, and emerging technologies. His "site themes" give one a sense of where he is coming from: "Aggregation Sciences," "Blogo-Eroticism and Other Hype," "Open Source, Open Content," "Tools, Toys, and Technology (Oh My!)" to name just a few. His Blogroll has some of the often-featured blogs, but also a few unique ones - Surfin' Safari http://weblogs.mozillazine.org/hyatt/ and Chatterbots http://chatterbots.blogspot.com/ are wonderful. Marketing Brain-Flash: Icons for the Blog-eroticism and other eye-catching categories.
e-learning centre - http://elearningcentre.typepad.com/whatsnew/ Jane Knight's e-learning Center has been operational since 1994. The site is well-organized, functional and loads quickly (although some of the ads on the portal page are a bit distracting). The practicality and functionally of this site cannot be overstated. It is marvelous - the little things really do count. For example, I appreciate the "Reviewer's Note" at the end of each article lead-in. It is one of the best sites I have seen for individuals who either need to organize an e-learning program or completely overhaul the one they have. She provides links and information to guide individuals along each step of the way. Granted, when perusing her site one gets the sense of a giant box under the Christmas tree labeled, "Some Assembly Required" - but, thankfully, you're not alone. If you're lacking the proper tools or the expertise, consulting services are available as well. Marketing Brain-Flash: Jane has a very friendly presence. We want more! More pictures of Jane working on projects, with clients, leading workshops.
Four more e-learning blogs profiled, ranging from the chatty and personable Michael Feldstein's e-literate to Jane Knight's e-program lifesaver, the e-learningcentre, to overwhelmingly encyclopedic e-learningguru and elearnopedia. All are outstanding and a huge service to the community. They encourage the sharing of knowledge and collaborations. Now, if I could only keep my "marketing brain-flashes" to myself (!). Oh well.
elearnopedia -- http://www.elearnopedia.com/Truly outstanding resources are available here. The articles on effective online instruction are both well-researched and accessible. The articles are neatly arranged by category: Academic Researchers, Educators, Developers, General Interest, and Content Managers. The articles are easy to access and useful. For example, in "Educators," I found Marc Miller, Kelly Rainer, and Ken Corley's "Predictors of Engagement and Participation in an Online Course" to be very useful, particularly from a behaviorist perspective, with a view to self-efficacy, motivation, and perceived relevancy of course materials. It did not measure the perceived difficulty of access or comfort level with modes of content delivery, which, although it would have complicated the picture, would have incorporated a realistic factor. I would be interested in seeing how entertaining, aesthetically pleasing, or otherwise semiotically compelling icons and other visual elements promote engagement. A major flaw: The citation does not appear in the portal, nor does the URL to the direct link. Marketing Brain-Flash: Slug-fest it! Add slugs (citations, URLs, author names).
e-learningguru -- http://www.e-learningguru.com/dashboard.asp This is an amazingly content-rich site, with numerous pages and nicely organized sets of links. In addition to links to articles, weblogs, and white papers, there is a useful glossary. There is a bit of information overload, and I would enjoy seeing how our wise elearning guru (Kevin Kruse) would rate the articles - perhaps not stars, or otherwise pedestrian qualitative evaluations, but by yoga poses to indicate how it makes the reader stretch. Kevin Kruse's "Blog Roll" features unique and often-overlooked weblogs dealing with all aspects of elearning. Now, what are Kevin's thoughts and insights? I, for one, would love to read a daily op-ed piece or analysis. Marketing Brain-Flash: Kevin, tell us what you're thinking!
e-literate - http://mfeldstein.com/ Michael Feldstein's e-literate has everything I love in a blog - a sense of the person behind the blog, an engaging tone, great links, articles about all sorts of things, articulate commentary, and lively ways of classifying and thinking about e-cognition, e-learning, and emerging technologies. His "site themes" give one a sense of where he is coming from: "Aggregation Sciences," "Blogo-Eroticism and Other Hype," "Open Source, Open Content," "Tools, Toys, and Technology (Oh My!)" to name just a few. His Blogroll has some of the often-featured blogs, but also a few unique ones - Surfin' Safari http://weblogs.mozillazine.org/hyatt/ and Chatterbots http://chatterbots.blogspot.com/ are wonderful. Marketing Brain-Flash: Icons for the Blog-eroticism and other eye-catching categories.
e-learning centre - http://elearningcentre.typepad.com/whatsnew/ Jane Knight's e-learning Center has been operational since 1994. The site is well-organized, functional and loads quickly (although some of the ads on the portal page are a bit distracting). The practicality and functionally of this site cannot be overstated. It is marvelous - the little things really do count. For example, I appreciate the "Reviewer's Note" at the end of each article lead-in. It is one of the best sites I have seen for individuals who either need to organize an e-learning program or completely overhaul the one they have. She provides links and information to guide individuals along each step of the way. Granted, when perusing her site one gets the sense of a giant box under the Christmas tree labeled, "Some Assembly Required" - but, thankfully, you're not alone. If you're lacking the proper tools or the expertise, consulting services are available as well. Marketing Brain-Flash: Jane has a very friendly presence. We want more! More pictures of Jane working on projects, with clients, leading workshops.
e-Learning Blogs This Month: Part I
by: Susan Smith Nash
This is part 1 in a series which profiles the content, scope, and orientation of e-learning blogs that catch my eye. These largely unheralded resources are absolute treasure troves, mainly due to the insights and comments of the blog owners, who create open communities of practice with their willingness to share, exchange, and (sometimes) rant. I've noticed a disturbing trend, however, inasmuch as fewer blogs link to other blogs, and syndication is, in some circles, becoming a thing of the past. Why? That might be worth exploring.
Stephen's Web OL Daily http://www.downes.ca/Two recent articles, "Assessing Discursive Writing" and "Barbarian Inventions" keep readers on their toes. What differentiates Stephen Downes from other bloggers is the way he does not shrink from probing the underlying ethical and epistemological issues as he deals with topics as diverse as online registrations, grading and evaluating essays, and the way the mind processes signs and symbols. Although they may seem disparate, the thread that holds these together is a concern for how the small things impact the big picture. Further, Downes follows a clear sequence of cause and effect to demonstrate how something seemingly as inconsequential as an online registration may trigger an entire series of events, with negative impact. The rubric proposed in "Assessing Discursive Writing" seeks to re-establish consistency and fairness in what is often viewed as an utterly subjective enterprise. It might be useful to take a look at how MY Access and other automated essay scoring programs develop rubrics for hand-scoring and thus "training" the program via artificial intelligence. Great links, great feeds, pithy and provocative comments.
Opossum http://www.opossum.ca/Great stuff from Opossum, located in Quebec City. Some of the items are "Les ENT dans le monde" http://www.cndp.fr/DOSSIERSIE/46/acrobat/03704111.pdf is a very valuable article, a great example of how future collaborations can take place, particularly across borders and between lesser developed and more developed nations. An example could be with the World Bank-financed education initiative (ties with the Chad-Equatorial Guinea trans-Africa pipeline), sharing could take place. Other examples in Francophone Africa could include cooperatives to facilitate the establishment of online educational initiatives in re-opening nations such as Libya.
Jay Cross -- Internet Time Blog http://metatime.blogspot.com// A good mix of personal reflection on the Internet, online education, and new trends, as well as homages to important moments; the passing of Cartier-Bresson, for one. My favorite article in the last few months was an absolutely fantastic article delineating the differences between Furl and Spurl. This article cleared up questions I had - and, made me a bit surprised to find it was such a straightforward issue. Extremely useful, with lively tone and engaging writing.
George Siemens - elearnspace -- http://www.elearnspace.org/blog/Thorough and eclectic blog, with insightful commentary by Siemens. The website, elearnspace.org is a very valuable resource with respect to evaluation, assessment, and strategic planning of one's online courses and initiatives. Siemen's view is holistic and integrative, with a view toward future developments and practical-level implementation. He has a particular talent for putting together effective diagrams.
Ray Schroeder - Online Learning Update -- http://people.uis.edu/rschr1/onlinelearning/blogger.html Schroeder is a professor of communications and director of OTEL at the University of Illinois, Springfield. Unlike other blogs that tend to focus on instructional technology, design, and administration, Schroeder incorporates a view of changing demographics and the underlying dynamics that are resulting in the widespread adoption of online learning, despite misgivings. Clear presentation, but would be nice to see a more complete index.
elearningpost - http://www.elearningpost.com/index.aspThis is one of my favorite e-learning blogs. It's amazing how many articles editor Maish Nichani manages to incorporate, with pithy and useful descriptions. According to the site, elearningpost provides daily links to articles and news stories about Corporate Learning, Community Building, Instructional Design, Knowledge Management, Personalization and more. This is quite true - the feature articles are good, too, but I wish there were more of them. Some tend to be quite specialized, which is good, but doesn't quite correlate to the level of the discussions (very lively!) and the articles and news stories.
Learning Circuits - http://internettime.com/lcmt/There is a wonderfully provocative article on having a portal. This is a timely discussion, coming as it does, on the heels of vociferous discussions on the ethical, privacy, and access problems associated with requiring users to register through a portal. They are willing to tackle issues that will undoubtedly influence the future of instructional design and useability; namely semiotics, approached through Wittgenstein. I was hoping for a more robust discussion - something from the grammar of visual design - but, it's a start.
The Learned Man! -- http://elearning.typepad.com/A unique perspective is provided by focusing on the Indian e-learning community. This is an area of great potential, with amazingly fruitful discussions and linkages available. Needless to say, the notion of open source and shared innovation is pushed to the limit here. A recent discussion addresses the shortage of trained instructional designers (and, in parallel form, the shortage of robust instructional design training overseas). The uncomfortable reality is that most formal ID programs are 3 - 5 years behind the times. Granted, fundamentals do not change; but in my opinion, it is pretty irresponsible of educational institutions to offer instructional technology and design courses that do not incorporate the latest research, trends, evolving needs, or software/systems. As in the case of video game designers, the best ones tend to be multi-disciplinary and self-taught, or receive training within the development company itself, which is an issue fraught with its own intellectual property & ethics dilemmas.
What Ralph Knows -- http://ralphpoole.typepad.com/ralphknows/Ralph Poole knows a staggering array of things - it helps, too, that he's been in the instructional technology, information technology industries for more than 30 years. His blog touches on almost all aspects, with a heavy emphasis on the technology side of the equation. I'm intrigued by his articles and reviews of skype.com internet telephony. Back in the late 90s, internet telephony was supposed to change our lives. What happened? His article makes me want to investigate. The links to Barbara Poole's online gallery are stunning. Her series of oil paintings are riveting -- my favorite is the "Saint" series. Brilliant narrative & reflexivity in the composition, the palette is evocative of Commedia dell'Arte.
This is part 1 in a series which profiles the content, scope, and orientation of e-learning blogs that catch my eye. These largely unheralded resources are absolute treasure troves, mainly due to the insights and comments of the blog owners, who create open communities of practice with their willingness to share, exchange, and (sometimes) rant. I've noticed a disturbing trend, however, inasmuch as fewer blogs link to other blogs, and syndication is, in some circles, becoming a thing of the past. Why? That might be worth exploring.
Stephen's Web OL Daily http://www.downes.ca/Two recent articles, "Assessing Discursive Writing" and "Barbarian Inventions" keep readers on their toes. What differentiates Stephen Downes from other bloggers is the way he does not shrink from probing the underlying ethical and epistemological issues as he deals with topics as diverse as online registrations, grading and evaluating essays, and the way the mind processes signs and symbols. Although they may seem disparate, the thread that holds these together is a concern for how the small things impact the big picture. Further, Downes follows a clear sequence of cause and effect to demonstrate how something seemingly as inconsequential as an online registration may trigger an entire series of events, with negative impact. The rubric proposed in "Assessing Discursive Writing" seeks to re-establish consistency and fairness in what is often viewed as an utterly subjective enterprise. It might be useful to take a look at how MY Access and other automated essay scoring programs develop rubrics for hand-scoring and thus "training" the program via artificial intelligence. Great links, great feeds, pithy and provocative comments.
Opossum http://www.opossum.ca/Great stuff from Opossum, located in Quebec City. Some of the items are "Les ENT dans le monde" http://www.cndp.fr/DOSSIERSIE/46/acrobat/03704111.pdf is a very valuable article, a great example of how future collaborations can take place, particularly across borders and between lesser developed and more developed nations. An example could be with the World Bank-financed education initiative (ties with the Chad-Equatorial Guinea trans-Africa pipeline), sharing could take place. Other examples in Francophone Africa could include cooperatives to facilitate the establishment of online educational initiatives in re-opening nations such as Libya.
Jay Cross -- Internet Time Blog http://metatime.blogspot.com// A good mix of personal reflection on the Internet, online education, and new trends, as well as homages to important moments; the passing of Cartier-Bresson, for one. My favorite article in the last few months was an absolutely fantastic article delineating the differences between Furl and Spurl. This article cleared up questions I had - and, made me a bit surprised to find it was such a straightforward issue. Extremely useful, with lively tone and engaging writing.
George Siemens - elearnspace -- http://www.elearnspace.org/blog/Thorough and eclectic blog, with insightful commentary by Siemens. The website, elearnspace.org is a very valuable resource with respect to evaluation, assessment, and strategic planning of one's online courses and initiatives. Siemen's view is holistic and integrative, with a view toward future developments and practical-level implementation. He has a particular talent for putting together effective diagrams.
Ray Schroeder - Online Learning Update -- http://people.uis.edu/rschr1/onlinelearning/blogger.html Schroeder is a professor of communications and director of OTEL at the University of Illinois, Springfield. Unlike other blogs that tend to focus on instructional technology, design, and administration, Schroeder incorporates a view of changing demographics and the underlying dynamics that are resulting in the widespread adoption of online learning, despite misgivings. Clear presentation, but would be nice to see a more complete index.
elearningpost - http://www.elearningpost.com/index.aspThis is one of my favorite e-learning blogs. It's amazing how many articles editor Maish Nichani manages to incorporate, with pithy and useful descriptions. According to the site, elearningpost provides daily links to articles and news stories about Corporate Learning, Community Building, Instructional Design, Knowledge Management, Personalization and more. This is quite true - the feature articles are good, too, but I wish there were more of them. Some tend to be quite specialized, which is good, but doesn't quite correlate to the level of the discussions (very lively!) and the articles and news stories.
Learning Circuits - http://internettime.com/lcmt/There is a wonderfully provocative article on having a portal. This is a timely discussion, coming as it does, on the heels of vociferous discussions on the ethical, privacy, and access problems associated with requiring users to register through a portal. They are willing to tackle issues that will undoubtedly influence the future of instructional design and useability; namely semiotics, approached through Wittgenstein. I was hoping for a more robust discussion - something from the grammar of visual design - but, it's a start.
The Learned Man! -- http://elearning.typepad.com/A unique perspective is provided by focusing on the Indian e-learning community. This is an area of great potential, with amazingly fruitful discussions and linkages available. Needless to say, the notion of open source and shared innovation is pushed to the limit here. A recent discussion addresses the shortage of trained instructional designers (and, in parallel form, the shortage of robust instructional design training overseas). The uncomfortable reality is that most formal ID programs are 3 - 5 years behind the times. Granted, fundamentals do not change; but in my opinion, it is pretty irresponsible of educational institutions to offer instructional technology and design courses that do not incorporate the latest research, trends, evolving needs, or software/systems. As in the case of video game designers, the best ones tend to be multi-disciplinary and self-taught, or receive training within the development company itself, which is an issue fraught with its own intellectual property & ethics dilemmas.
What Ralph Knows -- http://ralphpoole.typepad.com/ralphknows/Ralph Poole knows a staggering array of things - it helps, too, that he's been in the instructional technology, information technology industries for more than 30 years. His blog touches on almost all aspects, with a heavy emphasis on the technology side of the equation. I'm intrigued by his articles and reviews of skype.com internet telephony. Back in the late 90s, internet telephony was supposed to change our lives. What happened? His article makes me want to investigate. The links to Barbara Poole's online gallery are stunning. Her series of oil paintings are riveting -- my favorite is the "Saint" series. Brilliant narrative & reflexivity in the composition, the palette is evocative of Commedia dell'Arte.
Wednesday, August 25, 2004
The Ethics of Video Game-Based Simulation
Susan Smith Nash, Ph.D.
While video game-based simulation is gaining ground in interactive, multi-user distributed online learning, underlying ethical and philosophical issues remain unaddressed. These are, in some cases, quite troubling, because learner lives can be at stake. In other cases, profound issues dealing with instructional strategies and design are often only tangentially touched upon. This article identifies the ethical issues involved in using simulation in education and training, explores some of the conceptual and philosophical bases, and proposes ways to address the problems and adjust instructional strategies.
Specifically, this article examines the role of video game-based simulation used to prepare members of the military for combat situations. As opposed to the simulations of the past which focused on driving fast cars and piloting aircraft, the new generation of video game-based simulations focus on player-player interaction, and derive much from first-person shooter games. In contrast with the earlier versions of games, in which the goal was to accumulate enough points to go to different worlds or "levels," the new games combine the "sim" world-building experience, with customizable avatars to reflect the player's constructed persona, weaponry, and team members. Incorporating artificial intelligence, the game allows computer-controlled soldiers the ability to respond to student commands from a matrix of possibilities, each selection weighted via algorithms that build in probabilities and randomness.
Perhaps the most well-known of these video game-based simulations are the Full Spectrum Warrior and Full Spectrum Command, which were developed by the University of Southern California's Institute for Creative Technologies (ICT), employing the skills of Oakland-based Legless Productions' project management, Santa Monica's Pandemic Productions for the game development, and Sony Imageworks for the art assets. Not only have these two games received accolades from the Army; they have also become full-fledged commercial successes. Full Spectrum Warrior runs on Microsoft Xbox, while Full Spectrum Command is a PC-based game, accommodating multi-player as well as single-player settings.
The result the ICT initiatives and other such video game-based simulation projects is something that military training centers and training directors enthusiastically endorse. They cite improved training efficiencies, cost effectiveness, and a fine-tuned alignment of training with the actual job to be performed, particularly in terms the development of cognitive skills needed in highly specialized activities utilizing highly technical equipment.
For example, Brigadier General Frank A. Panter, Jr., Commanding General of the Marine Corps Warfighting Lab, points out what are viewed as indicators of quality: number of realistic scenarios that a student will experience; replicating the stress of the real situation; engaging the affect by being perceived as "fun" and "challenging." Gen. Panter states that the "MCWL is currently experimenting with the Combat Decision Range (CDR). The CDR uses facilitated, multimedia-driven combat scenarios to build artificial experience.
In a typical 45-minute training session, the Marine makes more tactical decisions than in a week in the field. CDR has been exposed to the operating forces through the infantry regimental training centers and other units. It is highly acclaimed by participants as a stressful, fun, challenging and very beneficial training tool" (Panter 2003).
He also points out that using video-game simulations as well as other types allows Marines to obtain a training experience of uniformly high quality, and to familiarize oneself with always-evolving high-tech equipment. Modeling likely scenarios and logistical challenges is also cited as an advantage of such training, as is the way that small unit leadership and fire team coordination are developed.
At this point, and important distinction should be made between video game-based simulation and the kinds of simulations employed by flight schools, air traffic control centers, and the military. One such simulation environment has gaming elements, but is large enough for individuals to move about in and develop the physical dexterity and cognitive skills they will need in the actual situation. Dubbed the MOUT, which is an acronym for Military Operations in Urban Terrain, the portable simulators fit in three shipping containers, with final dimensions of 8 ft wide by 9 ft high, and 20 ft long (Wilson, Nov 2003).
They have been shipped to various field training locations, as well as to forward locations, such as in Afghanistan. The simulators have equipment that generates smells, sounds, and sights (lasers, etc.), and allow students to monitor themselves via cameras, recording equipment, motion detectors, and other sensors that allow individuals to "score" themselves.
There is no doubt that training with large-scale, multi-sensory, role-playing simulation equipment is effective. However, truly distributed training is not available for everyone if there is a reliance on simulation equipment. Thus, PC-based simulations based on video games are viewed as ideal. In many cases, the video game-based simulation is used in conjunction with larger, full-scale simulation units.
For example, Full Spectrum Command was first used in conjunction with the Army's MOUT at Fort Benning, Georgia and classroom instruction (Wilson, Nov 2003). The students were asked to develop mission plans as a homework assignment, then to come to class prepared to discuss them and to input them into the game. Later, they emulated some of the conditions and contingencies in the MOUT's onsite large simulator.
However, when one moves into the area of PC-based video game-inspired multi- or single-player simulation, the issue becomes a bit muddy. Ethical issues abound.
Training Efficacy: Does gaming-based simulation really prepare people in the way they need to be prepared?
Perhaps the first and most obvious ethical issue is that of training efficacy. Obviously, it is unethical to claim that a video game is a simulation when it is not. A distinction between "simulation" and "scenario" must be observed, and the simulation must replicate lived experience in more than simply the visual. It must mimic the kinaesthetic by having similar controls. Further, decision-making must incorporate the same factors and conditions as the world that is simulation. For example, Full Spectrum Warrior contains a playbook with such features as "Crossing a Street Under Fire." This can be intriguing to a person who has been watching television footage of action in Baghdad, for example. It can seem like a godsend for a soldier who is about to be sent into a situation where he / she will have to cross a street under fire. Now, one has to hope that the instructions and the scenarios in the simulation are realistic.
Does it really happen this way? It had better do so, or lives will be lost.
The simulation claims to prepare the players for real-life situations by presenting the most likely scenarios, equipment, and avatars possible. To reiterate a point, to not do so is fraudulent and harmful to individuals who may experience physical consequences if they do not act appropriately.
* Equipment in the game must be as faithful as possible to the appearance, functionality, and uses (predictable and unpredictable) as what will likely be encountered in real life. The controls need to be as realistic as possible as well.
* The scenarios need to be as realistic as possible. They must be designed and tested by people who have actually been in the places involved.
* The avatars - the characters in the game - must be as realistic as possible, with appearance, actions, response times, and scripts as accurate as possible. The algorithms that control the "dialogue buckets" must have some randomness at the beginning, but ideally the system should "learn" the player through artificial intelligence.
Illusion of Randomness
It is vitally important that the players maintain the illusion of randomness so that the scenarios, actions, and other items expand the number of envisionable possibilities. The better simulations in the PC-game-based world are those that involve multi-player actions, and which are customizable.
Randomness, or at least a high degree of unpredictability, allows individual players to develop creative problem-solving skills and to make decisions quickly.
The problem with most games is that they feature "charms" and magical amulet-type pieces which, unfortunately, do not exist in the real world. The player who is used to being able to gain lives, power, or magical abilities by grabbing rings, flashing jewels, amulets, or other items, may experience a kind of cognitive disconnect when confronted with the real-life situation.
The other problem is that of the seductiveness of entertainment. The person who will spend hours playing an engaging video game, or who enjoys the game-inspired simulation may or may not make the connection to the real world. The pleasure and "reward" stimuli in the simulation / game environment are not likely to be the same in the real world.
Further, the real-world "rewards" - the social interaction and human bonding - employ different techniques in the real world. To take an extreme example, let's take a look at first-person shooter games. According to James Paul Gee, in the realm of the video game, "shooting" is "having a conversation" and a kind of "social interaction grammar" comes to the foreground. Needless to say, a first-person shooter game-based simulation and a live-fire practice activity employ different social interaction grammars. Shooting a person is not a good way to bond. It doesn't even send a very good message.
The ethical problems of mixing grammars are severe, yet no one seems to be addressing this. One cannot "translate" first-person shooter game to live fire and expect positive results. And yet, that is precisely what the intellectual model assumes when it proposes such a tactic for transfer of knowledge.
Invincibility is not merely entertaining, it invites hubris and/or mission-endangering grandiosity
To be entertaining and to trigger the fantasy-projection aspect of video games in order to have the player want to identify with the protagonist, the video game must feature main characters of superhuman flexibility, strength, and pain-free injury.
The characters must be very "Lara Croft" in order to "sell" - but what are the ethical implications of identification with an avatar that dies painlessly, accompanied by a somber musical signature. There may be an adrenaline-charged moment or two, but they are generally pleasurable. The gaming-based PC simulation does not incorporate thirst, hunger, nausea, pain, dizziness, shock, or other conditions under which a fighter must work in the real world.
Further, when a Lara Croft action figure is employed, the player identifies with the fantasy-qualities and loses sight of the real mission at hand. That is probably unavoidable, given that only with great difficulty is the simulation addressing problem-solving issues that are connected to something. In other words, the "situated learning" that occurs is only situated within the identity of the player; there are no actual tasks and embodied action (the memory of lived experience) does not factor in at all. Needless to say, the player becomes engaged in a fantasy identity construction activity, and pays little or no attention to anything else.
Translate from 3D to 4D in the real world
Navigating real highways and roads using Mapquest is probably on of the best "situated meaning" simulation activities on the Internet. More simulation games should employ the techniques used there. Note that the virtual world represented by Mapquest is at all times connected to a tangible reality that the player must interact with. It is clear at all times that the graphical depiction of reality corresponds to a tactile reality; and, that errors in decision-making have an immediate negative consequence (wrong roads, getting lost).
Abstract notions are rooted in metaphors for embodied experience (Lakoff 1987) and comprehension is grounded in perceptual simulations that prepare agents for situated action (Barsalou 1999).
While Kress and van Leeuwen and others suggest that naturalistic color, lighting, etc. "count as real" (Kress and van Leeuwen 2003), Mapquest's simplified graphical representation and the user's easy identification of that as having a corresponding reality in the real world seems to argue the opposite point.
In the case of warrior-preparing video-game based simulations such as "Full Spectrum Command," the very naturalistic color lighting, and perspective encourage identification and a belief (a suspension of disbelief) in the verisimilitude of the representation.
The Dangers of Dehumanization
Playing a well-designed video game is a pleasurable social experience. To be honest to the experience, a video-game simulation of war should be unpleasant, uncomfortable, and horrifying, as is war itself. However, to fill players with horror, fear, nausea and pain would probably be viewed as demotivating, and it would be difficult to have popular acceptance of the technique as an effective training tool.
Further, many have argued that video game-based simulations encourage dehumanization, by encouraging the players to view other players as objects rather than sentient beings. Oddly enough, this may actually be a benefit, particularly to those in the infantry who are on the front lines and will actually be called upon to see the person they kill (as opposed to artillery or aerial-delivered bombs). Killing is often requisite upon the ability to consider the person a "target" and to disassociate oneself from the action.
Undetected Obsolescence
The training efficacy of a simulation is a function of its relevance to real-world situations. In the world of asymmetric war, particularly in urban settings, the techniques and approaches constantly change. In fact, this is the key to tactical advantage. Thus, a game must be modifiable and adaptable in order to accommodate new equipment, tactics, and players.
Although the designers may shrink at incorporating ethically unsavory tactics and enemy actions, it is necessary to do so, particularly when there are attempts at disguise, entrapment, booby traps, and improvised explosive devises. If the designer does not accommodate the ever-evolving world, the simulation is worse that useless; it is dangerous.
SMEs are Not God, but they do create worlds, and if they get it wrong, you can die.
The quality of a scenario is only as good as its verisimilitude, and its ability to represent the cast, the conditions, and the equipment the players are likely to encounter in the real world. Authenticity and relevancy are key factors. The subject matter expert team needs to "ground truth" often. In designing the simulations, the project managers must ask questions: Who decides what the scenarios are? What are the decision-making points? How many options are presented? Does the system employ adaptive learning? When and how are modifications allowed?
These questions are vital to preserving the integrity of the experience and avoiding the temptation to focus on the entertainment value, rather than focusing on the most important decision-making, leadership, and physical skills to be developed.
Mentor No More: Forgetting how mentorship happens on the ground (trust, etc.)
Although video game-based simulations claim to develop team-building and leadership skills, this is not always the case. The temptation is always to "cowboy" the game and to indulge identity fantasies rather than to develop the skills that will lead to missions accomplished, lives saved. Mentoring is vital. Individuals need to be able to see a seasoned leader and to be able to emulate him or her. Further, trust is developed in the team, and through shared danger, rescue, and extended bonding. This is difficult, if not impossible, to achieve in the video-game simulation world.
Perhaps the most persistent ethical issue remains in the area of training efficacy. Other issues are briefly alluded to, and include access, integrity of the experience, dehumanization, and instructional design. This article assumes video game-based simulation for military applications for individuals who are already either active duty or reserve forces. It does not touch on the ethics of using an entertaining experience or satisfying social interaction to recruit individuals to enlist for an experience that is utterly devoid of the glamour, non-stop excitement, pain-free, and freedom from loneliness, anxiety, and discomfort of the game. Further, in a game, no one really dies.
REFERENCES
Aldrich, C. (2004). Simulations and the future of learning. San Francisco: Pfeiffer / John W. Wiley.
Barsalou, L. W. (1999). Language comprehension: Archival memory or preparation for situation action. Discourse Processes 28: 61-80.
Bruer, J. T. (1993). Schools for thought: A science of learning in the classroom. Cambridge, Mass.: MIT Press.
Churchland, P. M. (1989). A neurocomputational perspective: The nature of mind and the structure of science. Cambridge, Mass.: MIT Press.
Erikson, E. (1968). Identity, youth and crisis. New York: Norton.
Freire, P. (1995). The pedagogy of the oppressed. New York: Continuum.
Harris, P. (2003). SimsSimsSims - Simulation: The game is on. Training and Development October 2003: 46-51.
Kress, G., and van Leeuewn, T. (1996). Reading images: The grammar of visual design. London: Routledge.
McDonald, Andy. (2004). Moving to the forefront. Military Training Technology. 9 (3): July 19, 2004. http://www.mt2-kmi.com/articles.cfm?DocID=542. Retrieved August 16, 2004.
McPherson, M. and Baptista Nunes, M. (2004). The failure of a virtual social space (VSS) designed to create a learning community: lessons learned. British Journal of Educational Technology. 35 (3): 305-321.
Nicol, D. J. and Boyle, J. T. (2003). Peer instruction versus class-wide discussion in large classes: A comparison of two interaction methods in the wired classroom. Studies in Higher Education 28 (4): 457 - 473.
Panter, F.A. (2003). Interview. Military Training Technology. 8 (1): Jan 1, 2003. http://www.mt2-kmi.com/archive_article.cfm?DocID=223 Retrieved August 16, 2004.
Pelligrino, J. W., Chudowsky, N., & Glaser, R. (2001). Knowing what students know: The science and design of educational assessment. Washington, D. C.: National Academy Press.
Pinker, S. (1999). How the mind works. New York: Norton.
Poole, S. (2000). Trigger happy: Videogames and the entertainment revolution. New York: Arcade.
Rip, M. R., and Hasik, J. M. (2002). The precision revolution: GPS and the future of aerial warfare. Annapolis, MD: Naval Institute Press.
Shinn, S. (2003). The leader within: Be, know, do. BizEd. Nov-Dec 2003: 30-35.
Wolf, M. and Perron, B. (Eds.). (2003). The video game theory reader. Bristol, PA: Taylor & Francis.
Taylor, C. (1994). The politics of recognition. In C. Taylor, K. A. Appiah, S. C.
Rockefeller, M. Waltzer, and S. Wolf (1994), Multiculturalism: Examining the politics of recognition. Ed. By A. Gutman. Princeton, N.J.: Princeton University Press, pp. 25-73.
Wilson, J. R. (2003) Fixed urban training. Military Training Technology. 8: (4), November 19, 2003:
Yan, Z.; Hao, H.; Hobbs, L. J.; and Wen, N. (2003). The psychology of e-learning: A field of study. Educational Computing Research. 29 (3): 285-296.
DocID=270">
While video game-based simulation is gaining ground in interactive, multi-user distributed online learning, underlying ethical and philosophical issues remain unaddressed. These are, in some cases, quite troubling, because learner lives can be at stake. In other cases, profound issues dealing with instructional strategies and design are often only tangentially touched upon. This article identifies the ethical issues involved in using simulation in education and training, explores some of the conceptual and philosophical bases, and proposes ways to address the problems and adjust instructional strategies.
Specifically, this article examines the role of video game-based simulation used to prepare members of the military for combat situations. As opposed to the simulations of the past which focused on driving fast cars and piloting aircraft, the new generation of video game-based simulations focus on player-player interaction, and derive much from first-person shooter games. In contrast with the earlier versions of games, in which the goal was to accumulate enough points to go to different worlds or "levels," the new games combine the "sim" world-building experience, with customizable avatars to reflect the player's constructed persona, weaponry, and team members. Incorporating artificial intelligence, the game allows computer-controlled soldiers the ability to respond to student commands from a matrix of possibilities, each selection weighted via algorithms that build in probabilities and randomness.
Perhaps the most well-known of these video game-based simulations are the Full Spectrum Warrior and Full Spectrum Command, which were developed by the University of Southern California's Institute for Creative Technologies (ICT), employing the skills of Oakland-based Legless Productions' project management, Santa Monica's Pandemic Productions for the game development, and Sony Imageworks for the art assets. Not only have these two games received accolades from the Army; they have also become full-fledged commercial successes. Full Spectrum Warrior runs on Microsoft Xbox, while Full Spectrum Command is a PC-based game, accommodating multi-player as well as single-player settings.
The result the ICT initiatives and other such video game-based simulation projects is something that military training centers and training directors enthusiastically endorse. They cite improved training efficiencies, cost effectiveness, and a fine-tuned alignment of training with the actual job to be performed, particularly in terms the development of cognitive skills needed in highly specialized activities utilizing highly technical equipment.
For example, Brigadier General Frank A. Panter, Jr., Commanding General of the Marine Corps Warfighting Lab, points out what are viewed as indicators of quality: number of realistic scenarios that a student will experience; replicating the stress of the real situation; engaging the affect by being perceived as "fun" and "challenging." Gen. Panter states that the "MCWL is currently experimenting with the Combat Decision Range (CDR). The CDR uses facilitated, multimedia-driven combat scenarios to build artificial experience.
In a typical 45-minute training session, the Marine makes more tactical decisions than in a week in the field. CDR has been exposed to the operating forces through the infantry regimental training centers and other units. It is highly acclaimed by participants as a stressful, fun, challenging and very beneficial training tool" (Panter 2003).
He also points out that using video-game simulations as well as other types allows Marines to obtain a training experience of uniformly high quality, and to familiarize oneself with always-evolving high-tech equipment. Modeling likely scenarios and logistical challenges is also cited as an advantage of such training, as is the way that small unit leadership and fire team coordination are developed.
At this point, and important distinction should be made between video game-based simulation and the kinds of simulations employed by flight schools, air traffic control centers, and the military. One such simulation environment has gaming elements, but is large enough for individuals to move about in and develop the physical dexterity and cognitive skills they will need in the actual situation. Dubbed the MOUT, which is an acronym for Military Operations in Urban Terrain, the portable simulators fit in three shipping containers, with final dimensions of 8 ft wide by 9 ft high, and 20 ft long (Wilson, Nov 2003).
They have been shipped to various field training locations, as well as to forward locations, such as in Afghanistan. The simulators have equipment that generates smells, sounds, and sights (lasers, etc.), and allow students to monitor themselves via cameras, recording equipment, motion detectors, and other sensors that allow individuals to "score" themselves.
There is no doubt that training with large-scale, multi-sensory, role-playing simulation equipment is effective. However, truly distributed training is not available for everyone if there is a reliance on simulation equipment. Thus, PC-based simulations based on video games are viewed as ideal. In many cases, the video game-based simulation is used in conjunction with larger, full-scale simulation units.
For example, Full Spectrum Command was first used in conjunction with the Army's MOUT at Fort Benning, Georgia and classroom instruction (Wilson, Nov 2003). The students were asked to develop mission plans as a homework assignment, then to come to class prepared to discuss them and to input them into the game. Later, they emulated some of the conditions and contingencies in the MOUT's onsite large simulator.
However, when one moves into the area of PC-based video game-inspired multi- or single-player simulation, the issue becomes a bit muddy. Ethical issues abound.
Training Efficacy: Does gaming-based simulation really prepare people in the way they need to be prepared?
Perhaps the first and most obvious ethical issue is that of training efficacy. Obviously, it is unethical to claim that a video game is a simulation when it is not. A distinction between "simulation" and "scenario" must be observed, and the simulation must replicate lived experience in more than simply the visual. It must mimic the kinaesthetic by having similar controls. Further, decision-making must incorporate the same factors and conditions as the world that is simulation. For example, Full Spectrum Warrior contains a playbook with such features as "Crossing a Street Under Fire." This can be intriguing to a person who has been watching television footage of action in Baghdad, for example. It can seem like a godsend for a soldier who is about to be sent into a situation where he / she will have to cross a street under fire. Now, one has to hope that the instructions and the scenarios in the simulation are realistic.
Does it really happen this way? It had better do so, or lives will be lost.
The simulation claims to prepare the players for real-life situations by presenting the most likely scenarios, equipment, and avatars possible. To reiterate a point, to not do so is fraudulent and harmful to individuals who may experience physical consequences if they do not act appropriately.
* Equipment in the game must be as faithful as possible to the appearance, functionality, and uses (predictable and unpredictable) as what will likely be encountered in real life. The controls need to be as realistic as possible as well.
* The scenarios need to be as realistic as possible. They must be designed and tested by people who have actually been in the places involved.
* The avatars - the characters in the game - must be as realistic as possible, with appearance, actions, response times, and scripts as accurate as possible. The algorithms that control the "dialogue buckets" must have some randomness at the beginning, but ideally the system should "learn" the player through artificial intelligence.
Illusion of Randomness
It is vitally important that the players maintain the illusion of randomness so that the scenarios, actions, and other items expand the number of envisionable possibilities. The better simulations in the PC-game-based world are those that involve multi-player actions, and which are customizable.
Randomness, or at least a high degree of unpredictability, allows individual players to develop creative problem-solving skills and to make decisions quickly.
The problem with most games is that they feature "charms" and magical amulet-type pieces which, unfortunately, do not exist in the real world. The player who is used to being able to gain lives, power, or magical abilities by grabbing rings, flashing jewels, amulets, or other items, may experience a kind of cognitive disconnect when confronted with the real-life situation.
The other problem is that of the seductiveness of entertainment. The person who will spend hours playing an engaging video game, or who enjoys the game-inspired simulation may or may not make the connection to the real world. The pleasure and "reward" stimuli in the simulation / game environment are not likely to be the same in the real world.
Further, the real-world "rewards" - the social interaction and human bonding - employ different techniques in the real world. To take an extreme example, let's take a look at first-person shooter games. According to James Paul Gee, in the realm of the video game, "shooting" is "having a conversation" and a kind of "social interaction grammar" comes to the foreground. Needless to say, a first-person shooter game-based simulation and a live-fire practice activity employ different social interaction grammars. Shooting a person is not a good way to bond. It doesn't even send a very good message.
The ethical problems of mixing grammars are severe, yet no one seems to be addressing this. One cannot "translate" first-person shooter game to live fire and expect positive results. And yet, that is precisely what the intellectual model assumes when it proposes such a tactic for transfer of knowledge.
Invincibility is not merely entertaining, it invites hubris and/or mission-endangering grandiosity
To be entertaining and to trigger the fantasy-projection aspect of video games in order to have the player want to identify with the protagonist, the video game must feature main characters of superhuman flexibility, strength, and pain-free injury.
The characters must be very "Lara Croft" in order to "sell" - but what are the ethical implications of identification with an avatar that dies painlessly, accompanied by a somber musical signature. There may be an adrenaline-charged moment or two, but they are generally pleasurable. The gaming-based PC simulation does not incorporate thirst, hunger, nausea, pain, dizziness, shock, or other conditions under which a fighter must work in the real world.
Further, when a Lara Croft action figure is employed, the player identifies with the fantasy-qualities and loses sight of the real mission at hand. That is probably unavoidable, given that only with great difficulty is the simulation addressing problem-solving issues that are connected to something. In other words, the "situated learning" that occurs is only situated within the identity of the player; there are no actual tasks and embodied action (the memory of lived experience) does not factor in at all. Needless to say, the player becomes engaged in a fantasy identity construction activity, and pays little or no attention to anything else.
Translate from 3D to 4D in the real world
Navigating real highways and roads using Mapquest is probably on of the best "situated meaning" simulation activities on the Internet. More simulation games should employ the techniques used there. Note that the virtual world represented by Mapquest is at all times connected to a tangible reality that the player must interact with. It is clear at all times that the graphical depiction of reality corresponds to a tactile reality; and, that errors in decision-making have an immediate negative consequence (wrong roads, getting lost).
Abstract notions are rooted in metaphors for embodied experience (Lakoff 1987) and comprehension is grounded in perceptual simulations that prepare agents for situated action (Barsalou 1999).
While Kress and van Leeuwen and others suggest that naturalistic color, lighting, etc. "count as real" (Kress and van Leeuwen 2003), Mapquest's simplified graphical representation and the user's easy identification of that as having a corresponding reality in the real world seems to argue the opposite point.
In the case of warrior-preparing video-game based simulations such as "Full Spectrum Command," the very naturalistic color lighting, and perspective encourage identification and a belief (a suspension of disbelief) in the verisimilitude of the representation.
The Dangers of Dehumanization
Playing a well-designed video game is a pleasurable social experience. To be honest to the experience, a video-game simulation of war should be unpleasant, uncomfortable, and horrifying, as is war itself. However, to fill players with horror, fear, nausea and pain would probably be viewed as demotivating, and it would be difficult to have popular acceptance of the technique as an effective training tool.
Further, many have argued that video game-based simulations encourage dehumanization, by encouraging the players to view other players as objects rather than sentient beings. Oddly enough, this may actually be a benefit, particularly to those in the infantry who are on the front lines and will actually be called upon to see the person they kill (as opposed to artillery or aerial-delivered bombs). Killing is often requisite upon the ability to consider the person a "target" and to disassociate oneself from the action.
Undetected Obsolescence
The training efficacy of a simulation is a function of its relevance to real-world situations. In the world of asymmetric war, particularly in urban settings, the techniques and approaches constantly change. In fact, this is the key to tactical advantage. Thus, a game must be modifiable and adaptable in order to accommodate new equipment, tactics, and players.
Although the designers may shrink at incorporating ethically unsavory tactics and enemy actions, it is necessary to do so, particularly when there are attempts at disguise, entrapment, booby traps, and improvised explosive devises. If the designer does not accommodate the ever-evolving world, the simulation is worse that useless; it is dangerous.
SMEs are Not God, but they do create worlds, and if they get it wrong, you can die.
The quality of a scenario is only as good as its verisimilitude, and its ability to represent the cast, the conditions, and the equipment the players are likely to encounter in the real world. Authenticity and relevancy are key factors. The subject matter expert team needs to "ground truth" often. In designing the simulations, the project managers must ask questions: Who decides what the scenarios are? What are the decision-making points? How many options are presented? Does the system employ adaptive learning? When and how are modifications allowed?
These questions are vital to preserving the integrity of the experience and avoiding the temptation to focus on the entertainment value, rather than focusing on the most important decision-making, leadership, and physical skills to be developed.
Mentor No More: Forgetting how mentorship happens on the ground (trust, etc.)
Although video game-based simulations claim to develop team-building and leadership skills, this is not always the case. The temptation is always to "cowboy" the game and to indulge identity fantasies rather than to develop the skills that will lead to missions accomplished, lives saved. Mentoring is vital. Individuals need to be able to see a seasoned leader and to be able to emulate him or her. Further, trust is developed in the team, and through shared danger, rescue, and extended bonding. This is difficult, if not impossible, to achieve in the video-game simulation world.
Perhaps the most persistent ethical issue remains in the area of training efficacy. Other issues are briefly alluded to, and include access, integrity of the experience, dehumanization, and instructional design. This article assumes video game-based simulation for military applications for individuals who are already either active duty or reserve forces. It does not touch on the ethics of using an entertaining experience or satisfying social interaction to recruit individuals to enlist for an experience that is utterly devoid of the glamour, non-stop excitement, pain-free, and freedom from loneliness, anxiety, and discomfort of the game. Further, in a game, no one really dies.
REFERENCES
Aldrich, C. (2004). Simulations and the future of learning. San Francisco: Pfeiffer / John W. Wiley.
Barsalou, L. W. (1999). Language comprehension: Archival memory or preparation for situation action. Discourse Processes 28: 61-80.
Bruer, J. T. (1993). Schools for thought: A science of learning in the classroom. Cambridge, Mass.: MIT Press.
Churchland, P. M. (1989). A neurocomputational perspective: The nature of mind and the structure of science. Cambridge, Mass.: MIT Press.
Erikson, E. (1968). Identity, youth and crisis. New York: Norton.
Freire, P. (1995). The pedagogy of the oppressed. New York: Continuum.
Harris, P. (2003). SimsSimsSims - Simulation: The game is on. Training and Development October 2003: 46-51.
Kress, G., and van Leeuewn, T. (1996). Reading images: The grammar of visual design. London: Routledge.
McDonald, Andy. (2004). Moving to the forefront. Military Training Technology. 9 (3): July 19, 2004. http://www.mt2-kmi.com/articles.cfm?DocID=542. Retrieved August 16, 2004.
McPherson, M. and Baptista Nunes, M. (2004). The failure of a virtual social space (VSS) designed to create a learning community: lessons learned. British Journal of Educational Technology. 35 (3): 305-321.
Nicol, D. J. and Boyle, J. T. (2003). Peer instruction versus class-wide discussion in large classes: A comparison of two interaction methods in the wired classroom. Studies in Higher Education 28 (4): 457 - 473.
Panter, F.A. (2003). Interview. Military Training Technology. 8 (1): Jan 1, 2003. http://www.mt2-kmi.com/archive_article.cfm?DocID=223 Retrieved August 16, 2004.
Pelligrino, J. W., Chudowsky, N., & Glaser, R. (2001). Knowing what students know: The science and design of educational assessment. Washington, D. C.: National Academy Press.
Pinker, S. (1999). How the mind works. New York: Norton.
Poole, S. (2000). Trigger happy: Videogames and the entertainment revolution. New York: Arcade.
Rip, M. R., and Hasik, J. M. (2002). The precision revolution: GPS and the future of aerial warfare. Annapolis, MD: Naval Institute Press.
Shinn, S. (2003). The leader within: Be, know, do. BizEd. Nov-Dec 2003: 30-35.
Wolf, M. and Perron, B. (Eds.). (2003). The video game theory reader. Bristol, PA: Taylor & Francis.
Taylor, C. (1994). The politics of recognition. In C. Taylor, K. A. Appiah, S. C.
Rockefeller, M. Waltzer, and S. Wolf (1994), Multiculturalism: Examining the politics of recognition. Ed. By A. Gutman. Princeton, N.J.: Princeton University Press, pp. 25-73.
Wilson, J. R. (2003) Fixed urban training. Military Training Technology. 8: (4), November 19, 2003:
Yan, Z.; Hao, H.; Hobbs, L. J.; and Wen, N. (2003). The psychology of e-learning: A field of study. Educational Computing Research. 29 (3): 285-296.
DocID=270">
Subscribe to:
Posts
(
Atom
)
Blog Archive
- 2024 ( 5 )
- 2023 ( 7 )
- 2022 ( 17 )
- 2021 ( 13 )
- 2020 ( 15 )
- 2019 ( 25 )
- 2018 ( 25 )
- 2017 ( 20 )
- 2016 ( 28 )
- 2015 ( 35 )
- 2014 ( 24 )
- 2013 ( 28 )
- 2012 ( 34 )
- 2011 ( 31 )
- 2010 ( 23 )
- 2009 ( 35 )
-
2008
(
61
)
-
June
(
7
)
- Jun 27 ( 1 )
- Jun 19 ( 1 )
- Jun 17 ( 1 )
- Jun 16 ( 1 )
- Jun 12 ( 1 )
- Jun 09 ( 1 )
- Jun 05 ( 1 )
-
May
(
7
)
- May 26 ( 1 )
- May 22 ( 1 )
- May 15 ( 1 )
- May 11 ( 1 )
- May 09 ( 1 )
- May 05 ( 1 )
- May 01 ( 1 )
-
June
(
7
)
-
2007
(
27
)
-
December
(
13
)
- Dec 27 ( 2 )
- Dec 24 ( 1 )
- Dec 23 ( 1 )
- Dec 19 ( 1 )
- Dec 17 ( 1 )
- Dec 16 ( 1 )
- Dec 10 ( 1 )
- Dec 06 ( 1 )
- Dec 05 ( 2 )
- Dec 03 ( 1 )
- Dec 02 ( 1 )
-
December
(
13
)
- 2006 ( 4 )
- 2005 ( 5 )
- 2004 ( 14 )