Dgroups 2 – Are you ready ?

It is now about four months since DGroups Partner organisations started to prepare their groups for migration to the new platform, and a lot of “house keeping” has been done since then. Thanks to your hard work, most organisations and most groups are now ready. Overall, 67% of all groups on the original DGroups server have been confirmed. It is likely that many of the remainder are inactive. Is your group ready to migrate?

If the person responsible for coordinating DGroups at the Partner organisation which supports your group has not yet contacted you then please ensure that they know you are ready. To find up to date contact details click on “Contacts” above.

Mark Hammersley

dgroups2 – the end of the beginning

On Tuesday 16th December Dgroups Executive Committee accepted our recomendation that the new platform is sufficiently developed for us to begin the migration process. We will be publishing more details this week and next on how the migration will take place.  In the meantime, the work of tidying up group ownership is almost complete.  The final step before we begin migration is some preparatory technical work with Igloo which we hope to complete by early next week.

(Note: this copy of an Escher print can be found all over the Internet but if there is an objection on copyright grounds to my posting it I will remove it)

Pete Cranston

dgroups 2 – 2 groups

A quick end of the week post to record progress on a significant milestone, in two ways.

1. The first group from Dgroups 1 was migrated across to the new platform at the end of last week. Hapee de Groot of Hivos, for it is a group that he manages, has been testing it with his colleagues. While there are some issues to be looked at, as you would expect, the important thing is that all the elements of the group came  across as planned: discussions, documents, members and their profiles- including passwords – were there when the group was opened. Great work, WA Research, and a necessary step in preparing for migration.
2. Saskia Harmsen of IICD had the bright idea of using a training workshop she was running in Burkina Faso as a place to both test and introduce Dgroups 2 in French. The result is at the top of this page. WA responded quickly, both to open the group and then – overnight ! – to amend and improve the French translations, following feedback. Saskia reported that, “the workshop participants and first users of the itrainers-fr group were really enthusiastic about the new platform”.  A big thanks to Saskia and the training workshop participants for their very valuable feedback!

We have begun the process of planning migration in more detail, and will be reporting on that early next week.

(NOTE:the Spanish and Portuguese translations are being worked on at the moment. )

Pete Cranston

Dgroups2 – milestones and progress

Here again is the dgroups interface, larger this time, because at least one reader found the previous screen dump too small. As you can see a number of people have continued to test the platform and make comments on features as they are added. As you would expect, people’s reactions are mixed. There are features which are seen as an improvement on dgroups 1 while others  are generating questions. All the comments have been logged – using the system itself – and will be reviewed.  Thanks again to all those who are putting in their time and patiently learning how things work.

Today, however, we reached a significant milestone. WA Research has completed the major part of the development they planned to meet the criteria set out in the contract. This means two things:

1. The core migration team has started more systematic testing of the platform, confirming what works and identifying issues to be looked at. We’ll keep you posted on progress.
2. We need to plan for an increase in the numbers of people – and groups – who are exploring and testing the new system. These preparations need to start with the people who create groups – called coordinators in dgroups 2.  We will be arranging briefing and discussion meetings with those people (called creators in dgroups 1) next week.

Feels a bit like Christmas, really: starting to wonder what’s going to be delivered.

Pete Cranston

Dgroups – a Bellanet sized hole

In 2002, Bellanet, IICD, OneWorld, DFID, and UNAIDS Southeast Asia got together to build Dgroups on top of some existing workspace applications that Bellanet had developed and were providing one by one to their partner organizations. For those  of you that don’t know, the original Dgroups platform uses two pieces of software: the Lyris mailing list manager plus ”front end” software written by Bellanet staff. The platform developed over the next three years with Bellanet staff and  contractors providing technical support to the 20 plus organizations that joined the partnership. Euforic was brought in early in 2006 to help coordinate the partnership and develop a roadmap for Dgroups future development. Since Bellanet Ottawa closed in 2008,  the platform has been technically hosted and supported by IGLOO in Canada, with Euforic still providing coordination and ICCO acting as administrator and banker.

Why on earth is this useful or important information?

1. “You don’t know what you’ve got ’til it’s gone”, as a Canadian once sang. Igloo and Euforic and a host of individuals across several Dgroups communities have done a great job in maintaining the platform since Bellanet Ottawa ceased operations. But the real contribution and value of the work Bellanet did, mainly behind the scenes, only became apparent when they were no longer there. A platform based on more than one software tool is always more difficult to manage, especially the relationship between the two tools. With their experience of building the platform, Bellanet staff knew how to tweak and manage it to keep it going smoothly. That has meant a steep learning curve for Igloo, which cost them a lot of time and energy – and many others disturbed weekends and sleepless nights. However, one of the members of the migation team, Sarah Kerr, is ex-Bellanet, which re-connects us directly to that experience and knowledge. For example, Creators and Administrators are in the middle of cleaning up the data on dgroups – identifying defunct groups, working out what to do with groups originally started by Bellanet, and so on. It was a relatively quick task for Sarah to run a mini-programme to reflect those changes – which she did on Monday 24th, so people should now see the changes they have made on the system properly. (Sarah is going to join us in blogging – and in case there is any confusion, this picture is the Canadian singer not Sarah).

2. User support for Dgroups has always started with administrators and group creators from member organisations. This included Bellanet for their own groups, but they also provided technical support and learning to administrators and creators. When the platform moved to IGLOO, this support role moved with it, but without the time to properly carry across that deep knowledge, which has created enormous challenges for all involved.

3. It’s useful as a basis for understanding the way Dgroups is changing. Firstly, the new platform is provided by one organisation, WA Research, which runs it as one integrated operation. Secondly, WA Research provide this platform already to 27 partners, and manages its development according to the needs – and budgets – of those partners. In that sense they operate like Bellanet, but on a different basis, since their income depends on the smooth operation of the system and they don’t have a whole range of other tasks and functions in the way that Bellanet did.

4. I haven’t seen anywhere in public, at least on the web, a public acknowledgement to the people who worked in Ottawa for Bellanet of the really good job they did for us with Dgroups. It took creativity, mountains of hard work and networking to build up the platform. I for one am very grateful, and want to say so in public. Thanks folks.

Pete Cranston.

Same, but different Dgroups

Here is a sneak preview of the new dgroups environment, alongside the dgroups we all know and love. The team and a small number of volunteers are experimenting with this early Alpha version. It is built from the existing ECS platform and, as we write, additional functions are being developed and added. Clearly, since it is based on an existing platform, there will be differences between the new and old environments but the target is to ensure that the functions – how we use the platform – will map from one to the other.

There is still some way to go in terms of development but as you can see a lot of progress has been made. The priority for development has been email functionality since, while ECS has a robust and elegant email platform, it operates differently to dgroups and it is crucial to the migration that the new platform supports the same email operations as the original dgroups.

We’re very grateful to the group that’s working with us on the site at the moment: finding your way around a new platform, while it is being built is sometimes as frustrating as interesting. Thanks too to the developers who are working hard to keep up with their comments and feedback.  While major development is in progress we are going to limit access to a small number of people but soon we will be ready to invite more reviewers.  If you are interested in being part of the next group please write to Mark Hammersley (mark.hammersley@gmail.com), telling us something about the dgroups that you are involved with.

Pete Cranston

Calling all Dgroup leaders

Last week we sent a message to everyone on the DGroup Administrators’ Announcement list.   It had 5015 email addresses subscribed.  You probably received a copy of the message.  If you did not see it, and if you are a leader, facilitator or administrator of a group hosted on DGroups, please go to www.dgroups.org/groups/administrators/ and join this group so that you are sure to receive future important announcements.

We have been looking more closely at who are the Administrators of the current dgroups.  There appear to be 4187 of us.  This analysis does not yet take into account which groups are active or inactive.  Also casual observation suggests some people use more than one email address.

Those who are good at math will notice that there are 927 email addresses on the Adminstrators’ Announcement List who are not currently administrators of DGroups.   There were also 702 bounces from our message to that announcement list.  These are going to require further investigation.

Some facts you might find interesting… and things we need to know

90% of dgroups have between 1 and 5 Administrators.  There are 117 groups with no Administrator.  Around 1% of groups have more than 10 Administrators, with one group having 419 and another 288.  The next largest is 51 then 47, 46, 32, 30.  We are contacting the coordinators who support these groups with large numbers of Administrators to better understand what circumstances lead to such arrangements

79% of Administrators are responsible for only one dgroup, with 3% being responsible for 10 or more groups.  The largest number of groups administered by one person is 173.  There are five Administrators who are each responsible for more than 100 groups.  8 of the top 10 Administrators are also DGroups Coordinators, working for organisations which are members of the DGroups Partnership.

Peer Support Group

We also have four Administrators’ Peer Support Groups (administrators-en, -es, -fr, -pt).  They have a total of 1402 members, with some people being subscribed to two or more of the different language communities.  Not all of the email addresses on these groups are registered as Administrators, and there are many Administrators who are not on these groups.  Most members of the peer support groups are also members of the announcement list.

Further discussion among group leaders will also take place, in English, Spanish, French and Portuguese, on the Administrators’ Peer Support Groups.  If you would like to join, please select from the following links and then click “join”

• DGroups Administrators Peers Support Group (English language)
• Administradores de Dgroups (Español)
• Administrateurs de Dgroupes (Française)
• Administradores Dgroups (Português)

Dgroups2 Migration: The Big Picture

Here’s the big picture of our journey to Dgroups 2.

Path to Dgroups 2 - keyword: continuity

Alpha phase – at present we are making sure the Dgroups 2 platform allows one to perform all activities one is used to. Dgroups is all about email: consequently most of our work now is about tuning the new platform to offer what is needed to continue smooth operation mailing list operation.

This is purely software development and testing operation: we work against a list of requirements for the new platform making sure we cover all aspects of important functionality of the existing one. The work is divided in three functional areas: email, web user interface, and user profile features.

Beta test – as soon as we complete all the major features, we’ll open up the platform for an increasing number of existing Dgroups users to look around, try new things, see how their groups will look like.

At this stage, the platform will have all major functions available, but those will be rough around the edges. We’ll continue polishing and working as you look around. We’ll all communicate a lot about what you like or not and whether we’re missing something crucial that the majority of you were able to do with the old system. During this period, we’ll provide you with a copy of your data from the live Dgroups site, but only to look at – you will still use the existing live site to run your groups.

If you are feeling good about what you see and are tolerant towards technology, go ahead and create a few new groups using the new platform. Sure, it won’t be 100% finished, yet it won’t be crashing either.

Transition – when we’re sure it all works well, we’ll ask you not to create new groups on the existing live site, but to use the new one. Existing groups might still function on the old site for a while, but all new stuff goes through Dgroups 2. At this stage, we’ll have full support in place, all email messages will pass through the new platform and continue to the old one – thus the new one will be a mirror of whatever is going on with the current platform.

We want to give you some time to check the new platform and look around, learn the basics without pressure – your important groups will continue to run on the existing platform you already know so well.

Switch – once you get to know enough of the new platform to send and receive messages, approve new members, add resources, …, we’ll just flip the switch and immediately the new platform will start sending messages instead of the old one. All messages will already be there, we’ll copy all resources in advance. No downtime.

The Great Beyond – on February 20, 2009, we’ll shut down the old Dgroups system forever. That day on, we’ll work hard on new and exciting things to make Dgroups the best place on the Internet for international development community to conduct their dialogue.

Looking into the current Dgroups

The DGroups home page today reports that there are 2674 groups, containing 112366 members and 41919 resources.  Last month we downloaded a complete set of statistics from the database, to better understand the story of DGroups and prepare for migration.

Which groups count ?

On 17th September 2008 the count shown on DGroups home page was 2612 groups, of which 1783 were listed in the public DGroups directory.  There were an additional 323 groups marked as “this is a testing workspace”, plus 339 more mailing lists without associated workspaces.   Some of the “test” groups and some of the mailing lists without a workspace appear to be in use.  That makes a total of 3274 groups for us to consider, in order to understand whether and how each should be transferred to the new system.  New groups, created since 17th September will also be included in the migration.

New groups and old

DGroups has grown steadily over the years.  Group members and leaders have different levels of experience with the current system.  We need to take this into account when planning for migration.  Those who are very comfortable with the current system may be reluctant to accept changes, even if introduced with the intention of making things better or easier for new users.

All shapes and sizes

Most dgroups are relatively small (median 15 members, average 66 members) but there are 34 groups with more than 1000 members and the largest group has nearly 12,000 members.   The migration team will be communicating with leaders of all groups over the coming weeks, to better understand their needs and to offer appropriate support as they prepare for the changes ahead.

Parlez-vous DGroups?

DGroups hosts groups in English, French, Spanish and Portuguese.  Most groups, members and messages are in English.  30% of all messages sent in the past 12 months were on the Spanish language groups.

Groups and Partner Organisations

The DGroups Partnership has 26 members and every group hosted by DGroups is supported by one of those Partner organisations.  Partners share the costs of running DGroups and assure it is used appropriately.  Some Partners have many groups, other Partners have few.  Bellanet, which originally created many of the current dgroups, has now closed and the remaining Partner organisations are working to ensure that all of groups formerly supported by Bellanet can continue to use DGroups, if they wish to.

Which Groups are Active?

So, how many of the current dgroups are active?  We have noted that some groups marked “test” have many members and many messages.  There are also groups with few members and which have had no messages posted for over 12 months.  The DGroups Migration Team is now working with each of the Partner organisations to identify which groups are active.  Groups which are inactive but which have important historical content (records of messages posted, resources shared) can also be moved to the new platform so that their history is preserved.

Mark Hammersley

Untangling the Dgroups threads

This stage of a project is a bit like preparing for an international trip: trying to collect together bits and pieces, addresses, contacts, passport, loose change…and do I need anti-malarials? The equivalent in this project is getting to grips with the current status of groups in dgroups. The international community who use dgroups are very mobile – in and out of organisations, in and out of communities and groups – and sometimes changing names and emails. As with all online communities, it is sometimes difficult to keep in touch with these movements. Our situation is made more complex because of the handover from Bellanet to Igloo, the organisation which now supports dgroups. Bellanet started dgroups, and pretty much kept up to date with the changes until they it was closed down last year. However it has been harder for Igloo who haven’t got the history, nor people who have experience of how Bellanet operated.

So our priority has been to fully understand the reality of groups – how many are currently active, how many are dormant (not currently active but likely to be in the future) and how many should have been closed but weren’t? I have to confess here, for example, that when I was at OneWorld several groups were started for projects I was involved in that shouldn’t be still be there – but still are because I didn’t make sure they were closed. There is also the important issue of what we are calling ‘orphan groups’, groups that don’t currently connect to a current member of the dgroups partnership. Bellanet, for example, set up lots of groups. Those ‘owned’ by IDRC transferred out of dgroups to an IDRC server but lots of groups, including some large active ones, aren’t currently linked to a member.

We are clear that all groups that want to migrate to the new dgroups will have the opportunity to do so which is why we are spending a lot of time at the moment sorting out these issues. We have started the process by contacting the creators of dgroups, who are working through the lists at the moment.

We are also keen to re-activate the peer-support groups that were busy in the past but have been less so recently. Peer-support is an important way to share knowledge and spread limited resources between larger and smaller communities. We have contacted all the listed administrators: we are placing bets on how many bounces there are.

Meanwhile, we have also been focusing on the email system, making sure the new platform can support the email functions that have always been – and will continue to be – a central part of dgroups. The four of us in the migration team are experimenting with a trial environment at the moment and, as planned, we will invite a small number of people to join us over the next couple of weeks.

Pete Cranston