I have fairly established opinions on this whole topic, which have gradually evolved over time; but I don’t think I’ve ever taken the time to actually articulate what they are. So for my own benefit I thought I might do that now.

That being said, this is only my first go, so I will most likely elaborate on things more over time.

Background

As a bit of a background here, I work in a central unit in the university, and one of the main missions of the team I’m on is to the Research, Evaluation and Development of technologies that can enable and enhance learning and teaching. As such we’re the R.E.D. Team in the TELT group, in Learning & Teaching @ UNSW (make sense?).

Now there is a lot of technology out there; a ton of tools, software and programs that do different things and MIGHT be useful for learning and teaching purposes. So it’s our job to identify the ones in particular amongst the diversity of the landscape that will be explored for potential release to the wider academic community, and then having identified potential candidates, work with central IT to establish pilot systems and conduct evaluations; and if all goes well, to transition them to enterprise level services.

What I mean by “centrally supported”

Now here I should point out, when I refer to the notion of something being “centrally supported” I don’t mean there is one ring to rule them all, and we hand our decisions down from on-high at which point they must be immediately implemented by the masses – and exploration of local alternatives is not allowed. I see technology in learning and teaching as being highly dependent on context – so I’m not the least bit interested in telling people how a tool must be implemented.

No, when I say “centrally supported” I’m referring to the more pragmatic, practical aspects – not the pedagogical ones. For example, the supporting infrastructure for centrally supported services are hosted and maintained by central IT – this includes the servers, updates, and tech refresh of the hardware. It also involves user and technical support when things go wrong, bugs are identified, or people are generally unsure of how to use the technology.

My unit does play a role in the professional development process, and facilitates discussions on how to actually use the technology to enhance the learning and teaching processess – but this is far more of a conversation between colleagues that aims to foster a culture of sharing in which individual instructors explore what the technology means to them and in turn shares what their experiences have been.

Pilots

The notion of pilots then is a very specific process that seeks to ensure that what is being released is

1. Of potential or recognised value to the academic community;
2. Technically reliable and scalable (e.g. works as you expect it to and can handle a load of potentially thousands of people);
3. Integrates fairly well with existing systems (HR, Student Admin, and others); and
4. Can be supported centrally.

DIY edtec

Again, this process seeks to identify what can be implemented at an enterprise level, not to dismiss or cast judgment on projects that are more strategic to localised groups. As far as I’m concerned, if you find a service that works brilliantly for your local purposes but isn’t necessarily useful to the wider institutions – go for it. The only consideration is that you’ll need to run it off of your own back – central IT can only do so much.

This is after all what I do with all of my web projects. This blog, YouTube, Twitter, SlideShare, Skype, etcetera – I look after them all myself and am responsible for sorting out my own issues. They work brilliantly well for my purposes and I’m keen to use them; but I don’t expect central IT to resolve problems or issues I run into when using them.

In many regards this is the essence of what DIY edtec is all about. In personal uptake lay a tremendous amount of flexibility and freedom to explore and use whatever you like, regardless of what other people think. You dictate your own destiny; but you also look after your own issues.

How this relates to pilots

Incidentally, I say all this to try and clarify what may appear to be a conflict between what I say about things like edupunk, and what I’m doing in the name of central services. Not everyone is interested in capable of running their own show from the standpoint of educational technology; that’s where centrally supported services come in. The issue with centrally supported services, though, is they have to cater to the needs of literally thousands of people. So by necessity the way they’re organised, implemented, and maintained is far more bureaucratic and convoluted.

This is particularly visible when it comes to pilots, since by necessity we have to be more detailed about planning, implementation, documentation and evaluation. In my use of DIY edtec, I am constantly evaluating the value of a platform as I go, based criteria that I alone dictate; but I’m just one person. When your userbase is potentially 30,000 people, what you pay attention to and the criteria you use to determine success or failure is monumentally more complicated.

Categories of Activity

So when it comes to pilots I personally look at several different categories of activity:

• Pedagogy – does the technology suit the identified needs of the learners and teachers; does it also offer freedom and flexibility to both instructors and students; can it support diversity in learning habits
• Technology – is the platform reliable; is there vendor support; in the case of open source, is there a reliable release cycle for patches and updates as well as robust community support; is there potential to integrate with existing institutional systems; do APIs exist to ease or automate administrative tasks.
• Maintenance - does the underlying infrastructure align with existing systems and expertise; how labor intensive is the maintenance and management
• Support - what are the support implications; is the framework very complex, which may lead to increased demands for assistance; does it have complex dependencies and requirements for end-user systems

Why pilots are important

While certain elements will be clearly apparent without introducing a technology to a user sample, there is a great deal that cannot be ascertained without introducing real-live people into the mix. This is why pilots are so important. They enable us to examine the true implications of a technology, and yet do so on the small scale under very controlled circumstances with clear boundaries for start and finish.

If the evaluation of the pilot comes back positive, then you can begin to introduce the technology more widely. However if the pilot is a disaster, the impact will be quite low relative to the problems with an untested enterprise level application. When things go wrong in a sample of a few hundred people, it’s difficult but manageable. When they go wrong in a population of 30,000 people, it’s disastrous.

Pilots are crucial because they expose potential frameworks to a systematic layer of research, evaluation and development, and thus ensure they are valuable, appropriate, and sustainable for use on the large scale.

More to contemplate…

This is just my first pass at articulating my thoughts on pilots; I expect to elaborate more over time.  In the meantime if I’ve overlooked anything glaring or you have your own experiences, research or resources to share PLEASE DO! I’m all ears.

Plan to throw one away. You will anyhow.

There’s a saying in the FOSS world that is something to the effect of “plan to throw one away. you will anyhow”. I believe this concept was originally proposed in Eric Raymond’s “The Cathedral and The Bazaar”, which discussed the devolved nature of open source development and has since became a hallmark piece of work within the movement. (Someone please correct my source if I’m wrong)

The idea suggests that the first version of a project won’t hit the target square on, because you don’t know what the real target is. Indeed, subsequent versions may be entirely different from the original objective in both form and function.

So effectively you don’t know what you’re trying to resolve or what need you’re trying to address until after you make the first attempt. Having done that you can evaluate, discover, uncover, discard and then refocus on the true need.

If this notion is true it requires a great deal of flexibility and an exploratory willingness to experiment with trial and error to get a software environment right.

However…

This is a relatively straight forward concept for standalone software development, but what are the implications when the software is slated to support a wider project, such as an online/offline community that is being developed in tandem?

I would argue there are a couple of fundamental choices that have to be made and hardwired into the planning and developmental processes, but after that the development process is relatively similar to the exploratory model outlined above.

The role of the individual

The first choice relates to the role of the individual in the usage landscape, particularly with regards to networks – and even more so with groups. Different models view the individual in different lights – and distinguishing them from the whole to varying degrees – and this affects all subsequent downstream activities in the online space.

Perhaps the best contrasting examples of this in the contemporary snapshot of the Internet are the notions of connectivism and collectivism. Very loosely stated, the differences in these ideas can be characterised by the difference between blogs and wikis respectively.

Blogs are highly individual spaces that clearly delineate the blogger/individual from those who visit the space. Commenting and discussion certainly take place in these spaces, however the place of the individual is ultimately at the top of the heirarchy, and their persona is clearly represented.

Wikis by contrast, are by nature a multi-user space in which content is aggregated together to form a unified, cohesive whole. Here the user is a means to an end; where the end is the gathering and sharing of a body of knowledge created by a group-mind. Many wiki engines support user profiles, however in the scheme of things they are a secondary or tertiary consideration.

This notion was highlighted in a different way by Stephen Downes in his video “Groups and Networks”. Downes argues that groups are inherently heirarchical, closed and rigid; networks are more equalised, open, and flexible. In terms of this discussion, groups place more consideration and emphasis on the priorities of the whole; networks on the those of the individual.

I would argue that Downes’ network model is more beneficial to the individual, as it provides them with space to come to their own conclusions, and to voice their own opinions and thoughts. It is also arguably more representative of the diversity in the community.

In the network model you hear a cacophony or chorus of different voices; in the group it’s more of a single voice characterising the overall consensus.

In terms of this discussion, the significance during planning is to establish who or what is at the centre of the model. A user-focused community will place the individual at the centre, and relate all activities in the environment back to the individual, via a unique user space such as a profile in a social network in the case of a centralised model; or perhaps a personal blog in a decentralised one. Alternatively, a group-focused community will place the group at the centre and relate all activities back to the group.

Centralised versus Distributed

The next key consideration is the nature of the community space. The idea of a “community space” can take many forms, not all of which involve a single online location – such as a wiki, social networking tool, or discussion forum. Increasingly a single community can span multiple environments and include several activity-specific sites such as those for image sharing (e.g. Flickr), rapid asynchronous discussion (e.g. Twitter, Jaiku, Identi.ca), blogs, discussion forums or all of the above.

The ultimate decision here can impact on several different topics, not the least of which include centralised versus devolved control, administrative considerations, activities, the nature of community interaction, and indeed what opportunities exist for the community members themselves.

[More to come as I continue to think this through...]

I’ve been experimenting with Facebook after a very long absence and have discovered that video is now available there. Having also spent some time looking for an Official Seesmic application for Facebook I’ve so far been unable to find one. Charles Cloughly indicates at least one does exist but says there’s nothing particularly substantial available currently.

Charles goes on to say that “he’s game” to develop a Facebook application for Seesmic, and knows there are several other API guys out there and asks “now that you’ve made the request for an application, what do you want the application to do?”

This presents a tremendous opportunity for Seesmic users – and even prospective Seesmic users. It’s not every day you’re given the chance to voice your thoughts on an application that doesn’t exist yet, and thereby inform its development; so let’s make this opportunity count.

I encourage everyone to give some thought to desirable functionality and make your wishes and thoughts known. If you’re a Seesmic user please send your thoughts as a video comment to the thread above. Alternatively please enter your thoughts in the comment area of this post and I’ll ensure they’re passed along (there may end up being a wiki somewhere for this purpose – so watch this space). If you’re not comfortably with leaving a comment then feel free to email me.

Update: Loic has indicated (see the comments area here) that they’re already looking into a Seesmic application for Facebook, saying “we are building one, should take a few weeks, we would love your thoughts about it!” This is all the more reason to ensure you’re thoughts and input are heard by the developers, so jump into the Seesmic thread or leave a comment here and I’ll pass it along.