[Note: this video only just updated, so quality will improve over time.]

As I’ve mentioned previously, an unfortunate reality of the current Twitter Search tool is the distinct – and short – lifespan for Tweets. After a matter of a few weeks content ceases to appear in the search results, effectively wiping them from the face of existence.

In my mind this is an absolute travesty. An enormous wealth of information is freely shared every day and is worth preserving for posterity – not only from the standpoint of access to links and resources, but also analysis of trends over time.

Therefore one of the pet projects I’ve been indulging in over the last year or so, Tweets in Perpetuity looks at how best to archive Twitter posts so they are searchable and easily referenced later. A number of people have asked how the system actually works, so I wanted to take the time to document it.

The basic building blocks of the system are:

• WordPress – http://wordpress.org
• FeedWordPress – http://wordpress.org/extend/plugins/feedwordpress/
• P2 Theme – http://wordpress.org/extend/themes/p2

In the case of P2 and FeedWordPress, both of these elements can now be installed directly from the WordPress Dashboard area. Please see the WordPress Docs area for information on Installing Plugins and Adding Themes.

Finally, a major shout-out goes to Jim Groom, whose work with P2 at UMW Blogs inspired this whole idea and yet I failed to acknowledge in the video.

This video was initially intended as a brief demonstration and update for a few people to help bring them up to speed on the R&D I’m currently engaged in, however it ended up comprehensive enough that I thought I’d share it here as well.

This clip is best viewed full screen with the HD setting selected. Press play to start the video, and then from the up arrow just beside the full screen button select 720p HD.

One of my many hats above and beyond that of educational technologist for UNSW is as pseudo-webmaster for the local homeschooling community that my family is a part of here in the Blue Mountains. As with most networks – educational or otherwise – we’ve had a need for a web presence to help organise our activities.

WordPress has been used to establish the first incarnation of the community website, and has been a very valuable tool to help facilitate things, however increasingly the logistics, planning and scope of the group activities and families involved has become more involved, thus necessitating a site redevelopment.

With this in mind I’ve begun to experiment with BuddyPress in earnest recently. BuddyPress adds a social layer to any WordPress installation, and opens the door to far more participatory elements than the base installation alone. This can include creation of groups, forums, file storage, messaging, and a wide variety of other activities.

When combined with the newly discovered and AWESOME Events Calendar plugin, this has helped establish (at least in principle) a formidable web presence that has been extremely easy to set up.

Given the potential use of this model to other online communities and/or educators, I thought it would be worth sharing what components have gone into this particular installation (as demonstrated in the above walkthrough).

So far this site incorporates the following plugins, all of them downloaded and installed from the WordPress Dashboard via the Add New Plugin option.

• WordPress 3.0
• BP Group Documents
• BP Group Management
• BuddyPress
• BuddyPress Announce Group
• BuddyPress Group Email Subscription
• BuddyPress Restrict Group Creation
• Clean-Contact
• The Events Calendar

At this stage it’s unclear how much of this will comprise the final version of the site, however I’m impressed enough with BuddyPress’ power to want to share this information more widely.

In continuing to ponder the topic of this blog’s evolution, I’ve decided to start to leverage this space as a content management system far more than I ever have before.  So I’ve begun to collate some of the work I’ve done elsewhere on the web into more of a hub using a series of sections, pages and sub-pages.

If you look at the menu structure, you’ll now see a few new headings.  Most notably these include two of the workshops I’ve facilitated this session on blogs and wikis.

The workshops that inspired this content have already taken place, however I’m keen to offer more of them in the future and will more than likely incorporate a similar model next time.  I also wanted to share what I’ve been up to with other people in case anyone finds a use for the material or resources.

As always, the content is shared under a Creative Commons License, so if you have a use for it feel free to appropriate whatever you need.

I’m also attempting to demonstrate how WordPress can be used as much more than your typical blog.  The blogging element is really valuable; don’t get me wrong.  But the power in WordPress is actually realised when you look beyond the blog.

This afternoon I’m facilitating the 3rd workshop in a series of sessions for the School of English, Media and Performing Arts covering blogs and wikis and how they can be used in learning and teaching.

After so much time spent this session battling learning management systems, it’s an understatement to say how incredibly excited I am to be working in more open, collaborative mediums.  So it’s going to be a real treat for me – hopefully the other participants will feel the same.

I’ve created a blog specifically for the blog component of the workshops, and wanted to share the link here in case other people can benefit from it.  It’s a basic example of how WordPress can be used as more than just a reflective space, but one that we will be using during the workshop and will act as a reference and/or discussion space later for anyone who needs it.

All the content on the blog is shared under CC-BY-SA, so if you have a use for the content, please feel free to adapt it accordingly.  Note I didn’t go with CC-BY because a few of the images I’ve used were licensed as CC-BY-SA and I wanted to respect the owner’s wishes.  Please see the image captions for licensing information.

Despite there being interest from outside the school on attending the sessions, these particular events are for EMPA Staff only – HOWEVER I’m more than happy to repeat the session elsewhere if people are interested.  Please leave a comment if you’re interested in organising something with me.

In the meantime, please visit http://fasstelt.newsouthblogs.org for the workshop blog – see the EMPA Blog Workshop section for the session plan, instructions and resources.

In the end I gave up. After many days of looking and searching – first for an easy solution and then progressively more and more complex ones as the week drew on – I ultimately realised I was developing an increasingly convoluted design merely to stay with a free public blogging service.

As it turned out this issue affects both WordPress.com blogs and Blogger/Blogspot blogs – there does not seem to be a solution in place for either platform. I have a sneaking suspicion this restriction is to combat spam abuse – because really it doesn’t arise from technical constraints. Services like Diigo have shown us how incredibly easy it is to set something like this up, the point is someone has chosen not to let us.

As proof of this, once I made the decision to go with a self-hosted installation of WordPress on my ISP, I had the solution up and running in a matter of minutes.

At a very basic level, all you need is a vanilla installation of WordPress (most commonly available through CPanel on your web host’s admin area), and a plugin like Twitter Tools, which is the plugin we ended up using.

Once these two elements have been installed it’s just a matter of configuring the Twitter Tool settings within the WordPress Dashboard area (using the Experimental Digest options), and you’re up and running.

What really sucks in all this is that the average Joe or Jane Blogger are unable to implement this sort of system because the business rules of the blogging providers are getting in the way.

I should pre-empt all this by saying that I am not a programmer.  I’m good at troubleshooting technical problems based on observation, trial and error – but I’m not a programmer.  So there may have been a far easier way to resolve this problem – as well as a more technically appropriate solution – however this is what I experienced, and what I did to fix it.

Suggestions on better solutions are wholeheartedly welcome.

Background

By design, the site incorporates a large number of pages (not posts).  In this sense we’ve used WordPress as more of a content management system (CMS) than a blogging platform.  Having used Drupal in the past, I find WordPress quicker to set up and easier to maintain, and thought it might be sufficient for this particular project – especially given the fact the primary users were relatively inexperienced technical users. (I still love Drupal though, no doubt about it.)

By and large WordPress has indeed been a great solution, however due to the nature of the number of pages (over 20), using the default menu option just wouldn’t have worked.  So we instead opted to go with the Multi-level Navigation Plugin.

By and large for the last few months it’s been a fantastic plugin that’s done everything we need it to.  However all of a sudden within the last little while it stopped working, and I wasn’t sure why.

Variables and Uncertainties

The fact is I’m not sure exactly when it stopped working, which posed some major challenges in the troubleshooting of the issue, since I’m not the primary user, moderator or administrator of the site and am not sure what changes may or may not have been made recently.

I do know that the primary users are not ones to mess around with source code or WordPress Dashboard settings, run updates or install additional plugins – only add and edit content.  So while it is possible that someone changed something, I’d say it’s fairly unlikely.

That said though, I do not believe the issue resulted from a bug or fault with the plugin.  Truth be told I have no idea what’s caused the problem, only what the symptoms where and what I’ve done to fix it.

This is why I’m documenting this problem.  The root cause is unclear, and as such I didn’t find much information available on how to go about troubleshooting it. Only through sheer force of will did I manage to narrow down the search to the header.php file in the theme I’m using.

The symptoms

In short, the links in the menu were not clickable; nor could you browse down through the page hierarchy.  The first level menu items were displayed as you’d expect, but that was the extent of things.  It’s as if they were just static text.

And yet the plugin settings all checked out, and the pages all existed; plus you could load them in the browser if you knew the direct URLs.  The page links were also displayed properly in the page source.  Swapping back to the native/default WordPress menu also worked successfully.

Additionally Internet Explorer displayed the menu properly, but all other browsers on the PC didn’t.  These included Safari, Firefox and Chrome.

To confuse matters more, I attempted to reproduce the error by exporting the pages and content from the main site, importing them into a test site, installing the Multi-level Navigation Plugin and configuring the settings in the same way – I even used the same banner graphic.  However the test site worked perfectly across all browsers.

The workaround

In the end the workaround lay in the header.php file of the theme I was using (Irrigation).

During the set-up process for the plugin, in addition to activating the plugin you also have to add a single line of code to the header.php page:

<?php if (function_exists('pixopoint_menu')) {pixopoint_menu();} ?>

You then have to either remove the native theme menu code or comment it out using

<!-- [original code here] -->.

Both of which I’d done some time before.

For some reason in order to fix the problem this morning I had to precede the plugin’s code with two additional break tags – <br><br> – before the menu was clickable again.

In other words:

<br><br><?php if (function_exists('pixopoint_menu')) {pixopoint_menu();} ?>

Lo and behold, once I’d made that change the menu started working again.

Non-techie explanation

My current theory is that the banner graphic was obscuring the portion of the menu that recognised the mouse cursor and which was needed to show you the menu options, and then click on a page.  In effect, the menu didn’t know you were trying to bring up the navigation options because the banner graphic was somehow in the way (even though it didn’t look like it).

While this does resolve the issue – or at least provide a workaround – what it doesn’t do is explain why the menu stopped working in the first place…

What is Kaltura?

For starters, Kaltura is a comprehensive framework designed and developed to support the production, editing, hosting and delivery of online video. It is also open source, with the company structure based upon an open model.

Organisationally and in terms of process and activity, Kaltura is divided into two broad spheres of activity. Kaltura.org is the community wing, which is where many of the projects see their inception, development, and ongoing evolution. As with most open source endeavours, the projects are developed by a network of programmers, only some of whom work for the business side of Kaltura.

From what I’ve seen so far Kaltura is comprised of a variety of projects, including the server-side software necessary to host and deliver the media; web-based video editors used to produce new content, incorporate transitions and post-processing and create mash-ups of existing videos; plugins that enable integration with other open source platforms such as Drupal, WordPress and MediaWiki; and other supporting, peripheral elements like uploaders that facilitate the workflow processes.

In all cases, the software is available for download free of charge, thus enabling people to construct their own video hosting solutions withtin local IT infrastructure.

Kaltura.com is the second wing of the organisation, and generally speaking constitutes the business side of things. They use the developments that emerge on the Kaltura.org side in order to establish an enterprise level service for organisations who do not wish to maintain their own installations locally. This includes User and Technical Support, Service Level Agreements, and other agreements that guarantee base levels of operation and support.

Some of the income derived from the business activities is funneled back into the community activities in the form of sponsorship of open video groups such as the Open Video Alliance.

I believe this also helps fund the 10 GB’s of hosting space allocated to users of the Kaltura plugin for WordPress.

Roll Your Own Platform

One of the aspects of Kaltura that confused me initially was the question of where the video’s are displayed and therefore viewed. I had mistakenly expected to go to the Kaltura website and have access to a vast library of video clips in the way YouTube does it. In the wake of this morning’s webinar it’s become clear to me that this is not what Kaltura was created to do – or perhaps more accurately, it was created to let you roll your own video site.

My impression now is that Kaltura aims to provide all the necessary infrastructure required to upload, produce, and deliver online video – including the players, processors/transcoders and servers – however it does not appear to include (nor is it restricted to) its own native web platform. In other words, use of Kaltura must be accompanied by a platform or website within which to embed the players and/or widgets.

As mentioned previously, plugins are available for several popular open source platforms: WordPress, MediaWiki and Drupal.

Questions and Ideas

In finally gaining an accurate grasp of what Kaltura is, I’ve begun to consider how it might be used in an educational context. As it turns out, there is another Kaltura webinar on the horizon that will address this exact topic, so I’ll undoubtedly explore this idea more at that stage.

In the meantime some of the key questions that comes to mind is whether elements of Kaltura can be integrated into existing video solutions.

In the case of UNSW, development of UNSWTV has been ongoing for nearly a year now, and there are a number of obvious points of replication – such as processing/transcoding, hosting, streaming, and playback – where similar functionality isn’t necessarily needed. However personally I see a fair amount of potential for integration of the advanced editor – and perhaps the plugins for WordPress, Drupal and Mediawiki as well.

For example, if the advanced editor was linked to UNSW TV, students could use the tool to collaborate on the production of videos for assignments, assessments, student organsations, or other visual/creative activies – with the output of the collaboration then uploaded the existing central media servers and/or pass on to subsequent publishing points, therefore harnessing the existing functionality that has been designed and developed specifically for UNSW.

Likewise with the WordPress, Drupal, and Mediawiki plugins. These plugins supports a number of other options that expand the functionality of the native platform and could conceivable be of tremendous use if they integrated into existing infrastructure. From our standpoint it would make far more sense to use Kaltura in conjunction with existing infrastructure than add duplicate elements.

So what I’m interested in exploring is how (or indeed whether) these aspects of Kaltura could be added into the existing mix. That said, the UNSW TV project managers might consider these ideas to be out of scope, or technically prohibitive; that’s just where my head went this morning.

Components

As indicated, the syndication framework I’ve set-up incorporates WordPress as the blogging platform, FeedWordPress for aggregation and syndication, and a WP Theme called P2, which is designed as “a group blog theme for short update messages.”  Add in Akismet for spam protection, and WordPress.com Stat’s to “tracks views, post/page views, referrers, and clicks” and that’s everything.

From a technical standpoint it’s a very, very basic WordPress framework, however as I’ve discovered it works amazingly well for the task I had set out to achieve.

I’ve christened the site “Tweets in Perpetuity” and it is available at: http://p2.techticker.net/

Syndication

In terms of setting up the syndication elements, I’m currently experimenting with a couple of use cases – the feed for my Twitter updates, as well as the search results for #CCK09 and #ECI831.  The latter two feeds were important to test out, because I wasn’t sure exactly how WordPress would handle the fact the feed contained contributions from different authors.

As it turns out, the FeedWordPress settings give you control over a variety of different elements – including Authors, Posts, Categories and Tags.

For the Author setting, I’ve configured FeedWordPress to create a new WordPress account for Authors who haven’t been syndicated before.  This means it’s easier to filter out the contributions of different people so they’re not all lumped into one general category.  From the standpoint of attribution and citing sources this is very important – I don’t own the content and do not to wish have it appear as such.

Following on this logic, I’ve configured the permalinks to point back to the source on Twitter, rather than the local instance syndicated in the WordPress blog.

Categorising the Feeds

Due to the fact I currently have 3 different feed sources coming into the blog, it made sense to provide additional organisational structures  to differentiate one stream from the others.  To accommodate this I’ve set FeedWordPress to automatically assign each post in a feed to a specific category as soon as it’s syndicated and have used a term that is clearly related to the feed – in this case @mbogle, #cck09 and #eci831 respectively.

So now, using the category links in the right menu, you’re able to quickly filter the updates to one stream.

Search Indexing

One benefit of syndicating through WordPress that I’d failed to remember is the fact that, once the content has been pulled in and syndicated, it is also indexed and therefore searchable.  So now you’re able to take advantage of WordPress’s awesome full-text indexing and locate information far more easily that you can with the native Twitter search options.

Comment Fracturing

One factor that I’m still grappling with is how to mitigate comment fracturing, and ensure that any discussion that takes place on the WordPress blog is channeled back into Twitter and made available to the author of the content.

One of the settings in FeedWordPress concerns whether to allow commenting or not.  In the case of syndicated blogs it’s best to deactivate commenting and refer people back to the point of origin to engage in discussion there.

At the moment I have the commenting option activated, and people are allowed to post replies on the blog – however I’m in two minds about whether that should remain the case or not.

I spent quite a while trying to locate a plugin for WordPress that would enable people posting comments to also sent their comments to Twitter, but the one that looked promising does not appear compatible with the P2 theme I’m using.

The other option is to try incorporating a commenting framework such as Disqus, which includes this option as well – but again, I’m not sure whether P2 would be compatible or not.

Implications of Copyright on Syndication

Much more significant a topic than comment fracturing though is that of copyright, and the implications of copyright law on syndication frameworks such as this one.  I am not really all that cognizant of what the legal implications are of syndicating other people’s Tweets, and need to investigate this further.

For example, I’m uncertain about where fair use comes into play versus the need to seek approval prior to replication or reproduction of content.

If it is true that the moment a piece of content is created, it’s immediately copyrighted by the author, then theoretically I am violating the copyright of everyone I’ve syndicated so far who has not explicitly released their material under an open license.

For the moment what I’m going to do is include a note on the site providing my email address and requesting that anyone who does not want their content reproduced to contact me and I will ensure their material is removed from the site and not syndicated again in the future.

Recap and Looking Forward

All in all I’m quite pleased with this little experiment so far.  In only a few hours I managed to set up a syndication framework for preserving and indexing content that would otherwise have become harder and harder to locate, and indeed start to establish a more efficient means of filtering and organising the data.

The fact the framework runs regularly checks for new content means the system is largely automated, and appears to therefore require a relatively low degree of maintenance.

That said there are issues to contend with, in the form of legal questions of copyright versus reuse, comment fracturing and preservation of existing conversational threads – and it may turn out there are additional issues (or benefits) that I have not yet considered.  It is my hope these topics will become clearer the more I contemplate and experiment.

Digress.it is a plugin for WordPress that supports discussion at a far finer level than is traditionally available on most blogs.  Rather than limiting conversations to a single field at the bottom of a post, digress.it “lets you comment paragraph by paragraph in the margins of a text.”

I don’t necessarily see this model being applicable to all blogs and blog comments, however there are definitely instances where the model would be very handy.

Collaborative peer review and solicitation of feed back are two uses that come to mind.  The Australian Federal Government, for example, recently launched a Consultation Blog, which uses a similar framework based upon the Comment Press theme.  Their aim seems to be to subject papers and policy documents to as rigorous a feedback process as possible at as granular a level as possible.

The opportunity to engage in highly specific, in-depth conversations is relevant for other use cases as well – such as fine grained analyses, discussion and debate.

For example, a document available to the public domain or shared under Creative Commons license could be posted to a course blog and used as the focal point for an exercise.  In the case of particularly long documents you could even allocate different sections to different groups.  Then at the end of the discussion you’d have a single post (or persaps a small selection of posts) that contained the thoughts and opinions of an entire cohort of students, with the granular level of the discussion meaning the comments were organised into topical chunks.

In the case of both Digress.it and Comment Press, a Comment Overview panel is displayed to the right of the content and is kept in view no matter where the reader is in the screen.  This serves to keep the discussion right up front the whole time.

Just in the last two days I’ve decided to make some fairly sweeping changes to the commenting system on this blog and have installed the Disqus Comment System. This video is a basic overview of what Disqus is, how you set it up (primarily from the standpoint of self-hosted WordPress blogs), as well as a discussion on the implications that the framework has for readers as well as site owners.

Disqus describes itself as:

“Disqus, pronounced “discuss”, is a service and tool for web comments and discussions. The Disqus comment system can be plugged into any website, blog, or application. Disqus makes commenting easier and more interactive, while connecting websites and commenters across a thriving discussion community.”

How it works

Disqus establishes tremendous opportunities for both site owners/bloggers and readers/commenters alike in that it expands the available functionality far above and beyond the traditional text-based commenting structures found natively in most blogs.

Information from Disqus profiles is accessible from blogs and websites

This includes the option to link to personal profiles elsewhere on the web, connect with friends and colleagues and thus facilitate discussion, and ultimately retain far greater ownership over your comments than exists in traditional commenting frameworks since everything you say is aggregated against a profile on the Disqus website.  For example, this is my Disqus profile.

Network and Community Formation

One of the key outcomes of this framework is the opportunity for network development at both the blog level and the user level.  This is due to the fact blogs and users are equally represented in the framework and can thus become nodes in their own right.

As part of the set-up process each blog is given its own community area on the Disqus website (here is the one for this blog).  Not only does this enable users to quickly view and respond to comments (which are tied back into the blog), the community area also displays statistical information regarding Top Commenters and Popular Threads.  The increased focus on the individual in this framework means that readers are able to connect with one another – thus facilitating the development of networks and communities within and across blogs and sites.  Effectively the users start to become hubs for the discussion as much as sites are.

Synchronised Comments

Of critical significance for WordPress users (which forgot to mention in the video) is the fact that Disqus now synchronises comments between your WordPress blog and the Disqus site.  In the past when you replaced the native WordPress commenting framework with Disqus, all comments were stored on the Disqus website.  Personally this is the reason I never adopted it earlier; it seemed too great a risk to store the comments elsewhere.  With comments stored on both Disqus and WordPress, if you decide to revert to the native comments  at some stage – or otherwise need to deactivate the plugin – you are not going to lose all the discussion that took place while Disqus was implemented.

Many Platforms Supported

This is not to say that Disqus is restricted to WordPress.  The framework supports a variety of other platforms, including Blogger, Movable Type, Tumblr, and hosted WordPress.com blogs.  “Hacks” for other platforms have been developed as well.

There is a plethora of information available on the Disqus website so I encourage you to take a look if you’re interested, or take a look at the comment area of one of my posts here to see the framework in action.

Update: Apologies for closing comments on this post; it’s just been getting too much spam.