Report Summary

This report presents an evaluation of the Newham Online (NeOn) initiative, which was carried out through interviews with Newham Online partners and stakeholders. It pays particular attention to three related themes:

1. progress towards, and achievement of, its original goals;

2. the perceptions of NeOn, its structures and its modes of operation by the various stakeholders in the initiative and

3. options for future work.

1. Goals and achievements.

Newham Online has accomplished a great deal during its lifetime. In particular, it has:

· provided a vision of the development of Newham which promotes regeneration and at the same time raises the question of the ‘digital divide’, and thereby contributed to the enhancement of Newham’s reputation

· with the ‘extranet’ and the establishment of Newham.net Ltd., created the basis of a collaborative network infrastructure for the borough that has challenged the traditional reluctance of network operators to invest in ‘poor’ areas

· with the ‘gateway’, begun to draw together a set of resources and facilities relevant to people and organisations interested in services, communication and business opportunities in Newham

· provided valuable advice and consultancy for a number of groups and individuals in (and sometimes beyond) Newham

· begun to create a community of individuals and groups which can take forward the tasks of developing services, content and communications locally

Where NeOn has not achieved some of its originally specified objectives, this has often been because changes in the external networking environment have made these objectives redundant or lowered their priority. For instance, there were plans to use the extranet to host a large number of commercial web sites. This has become easily available, at low cost, since the Newham Online initiative began, and therefore there is less need for NeOn to promote this service. Other goals have not been achieved because of the reluctance of some organisations to share networking resources and services.

Newham Online’s limited resources have forced it to prioritise certain aspects of its original vision at the expense of others. In particular, infrastructure provision has been favoured over content building, the fostering of online communities and the publicising of the initiative. This has resulted in lack of knowledge about NeOn’s activities and achievements and an increasing ‘distance’ between NeOn’s core activists and some of its partners. These are issues that need to be addressed.

2. Perceptions of Newham Online and its Organisational Structure

Newham Online generally has an excellent reputation, as is also the case with one of its key partners, Newham Council’s IT Department. In particular, the vision, energy, helpfulness and skills of NeOn’s key activists were praised. Negative perceptions centred on a perception that NeOn had become a somewhat closed community, with a technical orientation that had the effect of excluding partners who were not in the central group or who were more interested in content and community-building than others. There was also some uncertainty about the nature of the relationship between Newham Online and Newham Council. Overall, respondents thought that Newham Online has given excellent value for the money that has been invested in it.

The loose, unconstituted structure of Newham Online was cited both as a source of strength and a significant barrier to further progress. The lack of formalised structures has allowed NeOn to move quickly on occasion and to communicate horizontally across institutional boundaries. However, this very structurelessness, combined with the lack of resources which concentrates the burden of work on NeOn’s Director and a small number of enthusiastic volunteers, limits the scope of NeOn’s activities. It also results in a tendency – one which is certainly not fostered deliberately – to restrict knowledge of NeOn’s activities to a small group of ‘hyperlinked insiders’ (with, as some people commented, an ethnic and gender composition untypical of the borough). Respondents noted the lack of NeOn-related meetings as a symptom of this unwanted contraction of the circle of active partners. They also noted the danger that key people could become overburdened or frustrated.

3. Future Options

This is a good time to consider changes, with progress having been made on many of the initial objectives and with the establishment of a formal organisation to run the initiative’s infrastructure projects. Respondents were clear that they wanted Newham Online to thrive and develop.

These was a high degree of consensus that more effort needs to be devoted to the building of a wider online community (or communities) and to the provision of content and applications. A general question raised by some people in relation to the future is how far the more commercial and the more community-oriented aspects of the initiative should be integrated – opinions here differed. With Newham.net Ltd. now set up to take care of the networks, Newham Online as a whole has a number of options:

1. Stay as it is, with a group of people elaborating the NeOn vision and trying to bring in other people and organisations. This was the least-favoured option among our respondents, although a couple of people felt that the lack of formal structures had enabled and encouraged their own involvement.

2. Reform itself into a more formal organisation, with appointed or elected representatives from local community groups, interest groups and organisational partners, perhaps with subgroups with specific roles and for different aspects of work, e.g. content, community development, fund-raising. Depending on the balance of objectives, this might bring in greater managerial expertise, more community involvement and/or an ability to delegate tasks and responsibilities. Some people, however, worried that such an organisation might become too rigid.

3. Create two separate organisations, one to be the representative organisation mentioned above, and another to be a kind of ‘think tank’ to concentrate on developing the broad vision of Newham’s online activities. The question here is whether the advantages of clearly separated functions outweigh the possible disadvantages of the creation of a divide between ‘thinkers’ and ‘doers’.

Whatever course is chosen, the consensus of the people consulted in this study was that Newham Online needs more resources, particularly in the form of extra paid workers, if it is to realise its vision and achieve its goals in the future. There is a clear need to raise or allocate money for this purpose.

The other point on which there was wide consensus is that whatever changes are made, if any, the support of the key actors in Newham Online, in particular that of the Director, is crucial. The Director would need to make a choice among the possible roles available within a more diversified organisational structure, and this choice would need to be the result of a collaborative and consensual process. There is no point in making changes that result in the loss of the skills and energy of the people who have created and developed Newham Online.

1. Introduction to the Report

This report presents an evaluation of the Newham Online (NeOn) initiative, paying particular attention to three related themes:

1. progress towards, and achievement of, its original goals;

2. the perceptions of NeOn, its structures and its modes of operation by the various stakeholders in the initiative and

3. options for future work.

Firstly, we examine both the original goals of Newham Online and its envisaged specific outputs, and assess how far these goals and outputs have been achieved. Where they have not been achieved, we assess whether – and if so how far – this is because of changes in the external environment, reprioritisation in response to experience, or other factors. It is recognised that the creation of online networks and services is a fast-changing area and that activities and priorities need to remain flexible in order to adapt to new opportunities and constraints.

Secondly, we evaluate how NeOn is perceived by its stakeholders. It is not always easy to specify exactly what NeOn is or where its boundaries lie. This is because it is a multifaceted initiative associated with a broad range of activities, and also because it does not have a tightly defined legal structure (although at least one part of it – Newham.net – does). This aspect of the evaluation examines both general perceptions and views on the organisation and procedures of NeOn.

Finally, we solicited opinions on what would be the most appropriate directions for Newham Online to pursue in the medium-term future. In this report, we set out a range of possible options and indicate where respondents show consensus or divergence.

Methods

We conducted the investigation in ways which are intended to be appropriate to both the themes outlined above and the timescales and resources available. The latter ruled out large-scale surveys and associated quantitative analysis. Therefore, we concentrated on the gathering and analysis of available material (text and online) and on interviewing relevant respondents.

We have interviewed a sample of people from stakeholding organisations which in the main represent three groupings:

1. infrastructure partners, e.g. Newham Council (including IT staff and partnerships office), Go2Find, Marconi;

2. training/education partners, e.g. the University of East London (UEL), Newham 6^th Form College (NewVIC), Newham Training Network (NTN), Letec, and

3. youth & creative partners, e.g. Newham Young People Online (NYPO), ArtOfChange, Stratford Circus.

We interviewed both people who are at the ‘centre’ of Newham Online (e.g. steering group members), people who are engaged with NeOn primarily as partners or enablers and people who relate to NeOn by virtue of working in related areas, not necessarily in Newham (e.g. East London Advanced Technology Training - ELATT). We decided not to conduct a formal interview with the NeOn Director, Richard Stubbs, as we had had extensive conversations with him during the planning phase and we wanted to report primarily on other people’s views of the initiative. All the interviews were semi-structured, i.e. they had some common questions while remaining flexible enough to accommodate the varying interests of respondents. A list of interviewees is given in Appendix 1.

We have decided not identify individual respondents when quoting them below, although we are aware that Newham Online, at least in part, is a fairly close-knit community and that inevitably readers will on occasion be able to hazard a guess at the identity of people quoted. While we have sometimes taken into account the organisational background of the people we were talking to when deciding which of their words to quote, we have not felt any great constraints on our reporting because of such considerations. We have been very pleased with the openness of our respondents, who communicated their interest in Newham Online and who made every effort to give us open, thoughtful, helpful and sometimes provocative answers and who were clearly talking as individuals rather than as institutional mouthpieces. Even those people who have been most closely involved with the progress of Newham Online did not noticeably speak from ‘defensive’ positions but made real efforts to convey both the strengths and the weaknesses of the initiative. We would like to take this opportunity to thank all the people who generously gave us their time, expertise and opinions, often in sessions which lasted significantly longer than the time we originally asked for.

To consider goals of the initiative we began by reviewing documentation on NeOn. Some of the material we have seen is quite specific about the general aims of the initiative and its envisaged outputs, and we made use of this material as a starting point for a discussion of how NeOn understands its mission.

To evaluate stakeholders’ perceptions of Newham Online, we selected a sample of organisations and relevant people within them in order to obtain a broad enough spectrum of views. Our initial classification of Newham Online stakeholders was as follows:

1. ‘core’ activists (which is a narrower group than the membership of the ‘core’ mailing list)

2. ‘group’ activists (moving forces with groups such as NYPO and the Creative Network)

3. Extranet partners (e.g. Newham College, Stratford Circus, UEL, Letec, Go2Find)

4. Funders/enablers (e.g. Newham Council, Marconi)

Some of these are easier to reach than others, and it was not possible to obtain a strictly representative sample from each, but we obtained views from all classes of stakeholder, with the exception of local politicians - because of timetabling problems - and, for purely practical reasons, of the class of potential users of future NeOn services. In the report that follows, we sometimes talk about people being closer to, or more distant from, the ‘centre’ of Newham Online activities. We freely admit that we have not measured this ‘distance’ with any degree of precision (e.g. through diagrams of who communicates with whom, when, how often, etc.). Instead, we have largely relied on our own observations and reading, and on respondents’ own evaluations of their positions.

The final theme – asking for views about possible future directions for Newham Online – did not require any special instruments of investigation. Views about NeOn’s future were included in the interview schedule and in many cases flowed fairly naturally from respondents’ answers to previous questions about goals, structures, procedures, etc.

The report was commissioned in mid-August, our initial proposal being dated 13 August 2000. Interviews were conducted during October, November and early December. Interviews were taped (but not transcribed) in order to enable a check on validity of notes taken.

The authors of this report have a long-standing interest in Newham Online, and recently made a proposal for a long-term research project to investigate the progress of the initiative from the viewpoint of the participative design of online services. While this proposal was not funded by the originally-envisaged sponsors, two UK research councils, we plan to propose long-term studies in the future. This smaller-scale study is intended to be of immediate practical relevance to NeOn and its stakeholders.

The report is structured as follows. First we provide a brief summary background to NeOn in terms of origins, stated goals and how the initiative is organised. This is followed by sections focusing on: NeOn’s goals and achievements; NeOn’s relationship with other organisations; how NeOn is organised; and NeOn’s options for the future.

Newham Online background

NeOn has been in existence for over four years. We do not propose to write its history here, but several respondents pointed to a seminar, held in 1996 and organised by Aston Charities, as the catalyst which (later) led to the formalising of the Newham Online initiative. Since then it has initiated various activities in pursuit of its goals (which are broadly concerned with the empowerment of local citizens and businesses through the use of telematics and the associated regeneration effects). These activities are at different stages of maturity, but the initiative has been in existence for long enough to make it useful to take stock of what it has done – for instance, one of the more important bids for funding (to the ERDF) specified a timescale for outputs ending in December 2000.

What exactly Newham Online is, or at least how it is understood and how it works, will be discussed in relation to our respondents’ contributions below. But, for orientation purposes, we give here a short list of key organisational features – or at least what we believe to be its key features. Submission of an early draft of this report exposed several misunderstandings on our part, some of which are shared by some of our respondents. The existence of such misunderstandings is itself an indication of differences in the way NeOn is perceived, especially the belief that NeOn is a more formal organisation than is actually the case.

Newham Online itself is not a formally constituted organisation, although since the end of 1999 it has had a ‘spin-off’ organisation, Newham.net Limited, which has formal company status. Newham.net, which may in the future seek to gain a Public Telecommunications Operator licence, manages the Newham Extranet and Gateway.

NeOn (as befits its informal status) directly employs nobody. Its Chair, Michael Mulquin, works on a voluntary basis. Its Director, Richard Stubbs, is a self-employed consultant who was paid up to March 2000 to direct the Extranet and Gateway project and who has since then been paid to work half time for Newham Online’s Affordable Broadband Connectivity project (with the possibility of future income from projects related to Newham.net Ltd). Other people who make significant contributions to its activities also do so on a voluntary basis, for instance as members of NeOn’s Steering Group. The founding directors of Newham.net Ltd are all members of NeOn’s steering group.

NeOn has a formal relationship with the Council of the London Borough of Newham in that funding for NeOn’s projects has been channelled through the Council, with the Council’s IT Director, Richard Steel, acting as ‘line manager’ for Richard Stubbs in relation to this funding.

Newham Online is involved in many different projects of various sizes. A list, taken from the NeOn website, is included as Appendix 2.

2. Perceptions, Goals and Achievements

In this section we discuss respondents’ perceptions of NeOn to give a useful context for identification of its goals and achievements – both generally and in relation to the central Extranet and Gateway projects. Goals and achievements of the respondents’ organisations are then discussed, as are respondents’ assessments of how far NeOn provides ‘value for money’. Respondents’ perceptions of the reputation of NeOn provides background for the section on NeOn’s relations with other organisations.

Respondents’ descriptions

As noted earlier, it is not always easy to specify exactly what NeOn is or where its boundaries lie as it is a multifaceted initiative associated with a broad range of activities. To provide a context for the views expressed by our respondents, here is the self-description of Newham Online as expressed in its main publication aimed at a general readership, the1999 Newham Online Report:

“Newham Online is a strategic partnership of academic, private, public and not-for-profit organisations working together to ensure that maximum benefit for Newham’s people is obtained from the development of information and communication technologies [ICTs].”

Recognising that this statement by itself is incomplete, the report goes on to describe NeOn in terms of a statement of vision and values – the key sentence being, “By the year 2005 Newham will be recognised as the leading UK centre for ICTs in terms of both their commercial exploitation and their use to benefit the local community” – and of its five main tasks which (in 1999) were, in brief:

1. Building the partnership and supporting local organisations

2. Harnessing resources and expertise to promote cyberspace development

3. Creating a Newham extranet which links the networks of NeOn’s major partners

4. Creating a Newham Internet gateway or portal to provide easy access to Newham-related information on the World Wide Web

5. Establishing Newham.net Ltd to operate the extranet and gateway and become a Public Telecommunications Operator

The multifaceted nature of Newham Online was reflected in the variety of responses we received to the question, “What is Newham Online? How would you describe it?” One of the funder/enablers characterised NeOn as a virtual organisation, an organisation which has to be perceived through its interactions. One of the activists gave it a material presence describing it as ‘a network of computers and people’. Others emphasised the partnership aspect of NeOn, or that of an organisation with a vision or with enthusiasm, but on the whole we can summarise perceptions of NeOn as falling within two broad classifications, those whose perceptions identify a community, and those who perceive NeOn as an enabler. The two perceptions are, of course, by no means completely distinct.

A number of people saw NeOn as a new community of interest in online activity, a group of interested people who meet online, and even a group of enthusiasts who are struggling against odds to develop and sustain something. Some partners perceived this community as a technical one. Examples of responses include: “It’s a classic e-community”; “It’s very virtual”; “It’s a bunch of enthusiasts”; “NeOn is a community network”; “NeOn is about building a community of strategic thinkers to create a common vision”. A few partners perceived NeOn as another community of sorts: a local partnership that works across sectors to bring everyone together in developing ways to make use of ICTs for regeneration, e.g. Newham.net becoming a telco. Responses included: “NeOn’s a local partnership which tries to exploit the benefit of the net for the Council and other community organisations”; “NeOn is a cross-sector partnership, with private sector funding as well as public sector (e.g. for the network audit)”; “NeOn is a partnership and an umbrella organisation”.

Those who perceived NeOn as an enabler identified it as providing a way of linking groups, of getting organisations to talk to one another, although most people noted that there was a long way to go. “NeOn brings people together through its vision” was one response; others included “Its main goal is to link the borough”; “It’s a way of linking lots of disparate groups”; “It pulls together expertise and enables dialogue, enables things to happen, links organisations”. Sometimes NeOn was identified as an enabler for organisations who use the net and want to get together to use it more effectively in communications. Some of the ways this view was expressed was via identification of it as an ISP or as a provider of online services for the council and other organisations. Hardware support for community organisations, providers of bandwidth, builders of infrastructure were all terms used to convey this enabling role. Examples of responses in this area are: “NeOn facilitates a cluster of ICT industries by bringing broadband and a foundation of ICT skills and knowledge into the borough”; “It wants to build a broadband extranet for all local organisations and residents”; “It aims to improve the digital infrastructure; “It’s an internet service provider (ISP), a provider of hardware support for community organisations”

In discussing NeOn with partners who were not active core members, there were sometimes questions regarding the structural relationship between NeOn and Newham Council (which will be covered below in the section on NeOn’s organisation). NeOn’s value as a forum for dialogue and local networking, both online and face-to-face, was identified as was its concern in providing various kinds of services in and around Newham electronically, i.e. its contribution to community development was clearly recognised. Its value in ‘putting the borough on the map’ was appreciated by many respondents, and its role as a provider of local consultancy was seen as one of its most immediate benefits. For instance, respondents said: “I get a good response from the ‘core’ [mailing] list – it brings together expertise and knowledge”; “I’ve learned a lot about technology, for instance about bouncing lasers off tower blocks”; “If an organisation asks [on the ‘tech’ mailing list] ‘how do we network our office?’ they get half a dozen quick responses, which saves time and money. Generally, lots of people get good, free consultancy from NeOn”; “The great benefit of NeOn is knowing where to go to get technical information”.

Some partners associated this enabler role with NeOn’s vision. Partners in training/education and media valued the strong vision of NeOn as an enabler for organisations wishing to use ICTs, but a number of people noted that the vision was technically driven. For instance, one respondent said, “One issue for NeOn is translating their ‘techie, toys for boys’ vision for other partners”. NeOn’s vision was seen to bring people together, but its technical focus was not enough to sustain continuing links between organisations and instead created an enclosed network of people driven by a technical agenda. A technical focus was pointed out not only in relation to vision, but also to the culture of NeOn, which was identified as assuming a shared technically-driven vision and technical expertise in its use of language: “Critics say NeOn is too technical”; “There used to be an ‘X-Files’ impression of NeOn: technical experts in the basement, nerds, odd people in a subterranean environment”. Generally, these views were expressed by people who are positioned further from the centre of NeOn’s activities, which reflects the differing priorities given to infrastructure and content/community-building areas in NeOn’s early stages.

People more centrally involved with NeOn recognised that this disparity in focus exists, or at least existed early on. Examples of this include, “Content has been neglected. However, NeOn has had to struggle with hardware and concentrate on other aspects”; “NeOn has been too focused on infrastructure and not enough on applications”; “NeOn has not got content generation off the ground”; “The balance is now changing from infrastructure to more content. But it had to begin with a ‘propellorhead’ focus – sort out the networks. You need the technical stuff first. It’s like transport – you need the roads before the cars”; “We’ve got a good foundation with the infrastructure, a proof of concept. We needed the infrastructure first”; “NeOn has good infrastructure: build the docks and the ships will come”. The focus on infrastructure was seen by some respondents to be a focus on market-related areas - seen as positive with regards to attracting new businesses to the borough and in keeping with the resources available - or more generally as a focus on the regeneration agenda, which was sometimes seen as separate from the parallel agenda for bringing different, perhaps less advantaged, groups together. This market orientation was viewed by some respondents as acting against an IT community vision, as an orientation in which the voluntary sector and small organisations cannot participate. As one education partner put it, “NeOn needs to begin to look at what happens at the end of the wires”.

While some partners saw the focus on bandwidth as appropriate, and indeed urged NeOn to go further towards commercial infrastructure provision, for others content was discussed as a problem and a concern. One activist noted that content served as a way to engage with the various communities within Newham and could enable wider access. Discussion of content was linked to the question of boundaries. In support of the current balance between infrastructure and content, one activist noted that content would be of interest only to the local community and go no further. A number of partners identified the need for links outside the borough, and in the case of new creative and training partnerships, links with other localities were seen to be desirable. More of the reasons for this early infrastructure focus will be discussed below in relation to the construction of the extranet.

Goals and extent of achievement

Newham Online’s goals can be broken down into the goals of the initiative itself and the goals that its main partners are trying to achieve. While we want to concentrate on the former, we recognise that because of NeOn’s partnership character it is not always easy to separate these from the latter. So we begin with a short discussion of the goals (specifically in relation to NeOn) of the partner organisations we contacted.

Goals for partners can be analysed under a number of headings, those of connectivity, technical support and technical opportunities, collaboration for specific projects, and regeneration and partnership projects.

Electronic connectivity to the outside world was a goal of many activists and partners and it was achieved for most of them. The emphasis on technical infrastructure was cited again and again in discussions with partners, and on the whole the infrastructure was seen to provide a good value way to achieve connectivity. For instance, “If NeOn weren’t there, we would need to fundraise or reduce our ambitions. We would need to get a 2mbits/s leased line costing somewhere around £25k per year”; “Networking around the borough, using lasers and microwaves, is a very cheap resource”. One respondent, whose chief concern was providing support for other local organisations, voiced the opinion that the connectivity provided by NeOn did not reach down far enough to encompass the many small, voluntary sector organisations in the area, although it was recognised that an attempt had been made to connect one such organisation (which had been frustrated because of a lack of a line-of-sight link) and that NeOn was not in principle opposed to reaching smaller organisations.

There were a few concerns expressed regarding quality of service. For instance, one respondent thought that the 2mbits/s link at the core of the network was now too small, and another highlighted the fact that connectivity was provided only on a ‘best effort’ basis: “NeOn is our ISP, but we don’t have a service-level agreement with them”. A number of the extranet partners and central activists expressed concerns about the lack of redundancy in the network: “Currently the extranet is fragile, non-redundant”; “I’m worried about lack of redundancy in the link – it’s fragile: pigeons and masking tape have both knocked out line-of-site links”, but it was recognised that providing such redundancy would be expensive in both time and money. These respondents were clear that if Newham.net is going to move the extranet beyond the ‘proof of concept’ stage into a full service, it will have to behave more like a commercial service organisation with formal service agreements: “We will need ‘five nines’ [= 99.999%] availability”. There are possibilities that if some current bids are successful, the network could be made much more resilient in the future.

Technical support was valued by partners as was the opportunity to make links with those who had technical expertise and could help in providing an understanding of what the available technology can do. A number of partners mentioned opportunities that had recently been announced and/or had become available, specifically the calendar and WebBoard. The calendar was seen to be very useful, but the few people who had tried WebBoard had experienced difficulties (mainly related to infrequent use, e.g. “I can never remember my password”). A number of partners expressed concerns about their lack of understanding of what the technology can offer and the tendency to talk in a technical language.

The opportunity provided for collaboration on projects such as ‘Connecting Creativity’, the ‘Creative Industries Telematics Project’ and digital artwork was valued. Regeneration projects and partnerships such as Three Mills were identified and the potential of a media industry cluster, a culture industry and the grid for learning were discussed. In terms of the regeneration goals of a number of partners, the media industry was identified as the most promising sector, and one where NeOn was making an active contribution in terms of brokerage activities and project management. One of the partner groups said that without NeOn there would be more need to network and bids would be more difficult to set up.

However, as mentioned above, there were concerns expressed about getting key people on board and translating a ‘technical toys for boys’ vision to a form that is more relevant to other partners. This issue was discussed in terms of developing a people network. A number of partners expressed concerns regarding wider access for people in the community and appreciated that NeOn was helping put real issues around community networks and access before the Council. Getting people round the table, then getting people from the table to action were seen as challenges still to be met. A significant point was made by one of the activists: when a suggestion is made, it is bounced back to the originator and they’re expected to do it. A situation of limited resources in which volunteers need to act as initiators and implementers of ideas can be a constraint on further development and place a cap on ideas and expectations. This will be discussed more fully in the section below on NeOn’s organisation.

Overall, NeOn has generated an awareness of the possibilities of ICTs for partners, raised issues of wider access within the LBN, and provided an infrastructure for local organisations, but those interviewed thought it still has a way to go in developing ICT community uses and in becoming the gateway for council regeneration. This can be interpreted as: NeOn has successfully met its goal by putting a mechanism is in place for ensuring infrastructure, or it can be viewed as stage one of a two stage development. Stage two would be the development of an IT community, one that might consist of more partnerships with local voluntary and community groups, a gateway/a “one stop shop for investment in Newham”, or a networked community that delivers a Council vision of electronic services.

Assessment of the Extranet and Gateway

Although we were asked to provide an evaluation of Newham Online generally, we were also asked to pay specific attention to progress in creating the extranet and the gateway, as these were the main constituents of a successful bid to the European Regional Development Fund, for a project which ended in December 2000 and which contained provision for an evaluation.

The Newham Extranet.

In the networking world, people commonly make a distinction between the Internet, an intranet and an extranet. The Internet is a worldwide network of networks whose owners agree to implement common standards and protocols and which provides services such as electronic mail, file transfer, access to information (most commonly using the standards and access software which make up the World Wide Web) and other facilities. An intranet is a single network, owned by a single organisation, which uses the same standards and applications software as the Internet but which is separated from the rest of the Internet, usually by a ‘firewall’ – a combination of hardware and software that keeps the internal network private. An extranet is created when two or more organisations agree to allow each other (some of) the privileges of being on each other’s intranet, usually be careful reconfiguration of each organisation’s firewall to allow members of the other organisation to be treated as if they were ‘insiders’. An extranet, then, is something more than an Internet exchange point, where networks exchange incoming and outgoing Internet traffic, because it requires an extra degree of trust between the extranet partners. How much trust is needed depends on the kind of access which is made available to the other organisation: this might range from an agreement to allow one organisation’s traffic to traverse the other organisation’s network on the way to a third party (which has security implications relating to areas such as junk e-mail and hacking) to an agreement to give the other organisation access to parts of a private, internal database.

The Newham extranet appears to provide different levels of interaction between different sets of partners. Mostly, it is there to provide routes for connectivity, e.g. a partner’s networking traffic might reach the rest of the Internet via Newham Council facilities, but there are also some examples of institutions providing privileged access to particular facilities (e.g. the University of East London Business School provided online learning facilities for Newham Council managers). Here we are less interested in detailing the facilities provided to the organisations using the extranet than in assessing how far the current extranet matches the original goals set by Newham Online.

A point made several times by different respondents was that it is necessary to remember that the situation with regard to the recognition and availability of the Internet is now very different from what it was during the genesis of NeOn. Two of the key enablers of NeOn pointed out that in the mid-1990s Newham was being neglected by the telecommunications companies. The borough was seen as being too poor to merit large investment: Stratford was going to be among the last areas for BT to convert to a digital telephone exchange, and the local cable company (at that time, Bell Cablemedia, now ntl:) had almost stopped developing the local franchise as it had low service uptake and a high proportion of bad debt. If Newham was going to overcome this disadvantage and use ICTs for regeneration, it (in whatever form) would need to make its own investment in infrastructure.

When NeOn considered this problem, it realised that it would be impossible to install comprehensive networking facilities for local organisations and, ultimately, individual citizens by itself, so it developed a strategy aimed at connecting the already existing networks of large local organisations such as the Council, the University, the College, etc. and saving what investment money that existed in order to fill in any gaps.

How successful has this strategy been? We took the document containing the original description of the ERDF extranet and gateway project, and extracted from the list of tasks those that could be easily assessed. The list, which we then showed to respondents for comment, was as follows:

1. Launch the extranet

2. Attract an adequate number of NeOn partners to the extranet

3. Deploy network applications across the extranet

4. Spread awareness of the extranet to Internet users in Newham

5. Promote ten ‘exemplar’ companies carrying out online trading

6. Host 200 companies with free Web presence within the extranet

We knew in advance that the last two tasks had not been carried out, mainly because the environmental conditions had changed considerably since the extranet project was conceived. The costs of setting up an online trading site, and the costs of having a web site hosted, have fallen considerably since 1997. Also, suitable e-commerce software has become more easily available. Today, therefore, there is less to be gained from implementing the last two tasks listed above. Reassuringly, none of our respondents claimed that these tasks had been carried out, and most did not know that they were part of the extranet project at all.

The amount that people knew about the other extranet tasks varied, not surprisingly, according to the rough distance from the ‘centre’ of Newham Online of the respondents. Almost everyone knew that the extranet existed, i.e. that it had been launched, and some people knew several organisations that were connected to it. However, outside the group of organisations that are actually using the extranet, knowledge is very limited. A selection of the responses illustrates this: “I didn’t know how far NeOn had got with the extranet until I talked to Richard Stubbs recently”; “The extranet is developing, but I’m not sure where it’s at”; “I wouldn’t know about it if I weren’t on the Steering Group”; “Lots of people, even those on the core list, don’t know what the extranet is”; “I’ve not looked closely at this [the extranet]; it’s for the technocrats. … I don’t know how much awareness there is of it.”; “I’m aware of it, but I don’t know any details”; “All the extranet partners know its value; non-extranet organisations might not”.

The above statements, particularly the last one, illustrate that the fourth task listed above, that of spreading awareness of the extranet to Internet users in Newham, has not been given a high priority. Newham Online either has not had the resources, or has not felt it appropriate, to market the extranet to a wider public. What has been done – with some success - is to target awareness of the extranet at a small(-ish) number of organisations which might be in a position to use it. As mentioned in a previous section, those organisations which are, in fact, connected to the extranet very much appreciate its value, although not every organisation which has been approached about the extranet has joined – we were given the example of one centre which was involved in negotiations but which, for the time being at least, declined to connect.

We did not have time to talk to ‘non-joining’ organisations to find out their reasons, but it is possible that some smaller organisations might have cost worries, while others may have worries about the security implications of granting privileged network access to other organisations. We know that the IT managers of our own institution are taking a cautious attitude to the extranet because of security worries and (perhaps to a lesser extent) because of the connection policies of the wider academic network (JANET) which provides its Internet connection. The extranet connects to the University of East London at one point, to provide the application discussed below, but extranet traffic is not allowed to travel to other parts of the University’s network although, according to one respondent, it would require only a small amendment “of one line of firewall code” to overcome this restriction. Some respondents pointed to a further reason for slow progress on the sharing of applications across the extranet: competition. As one person put it, “Different partners are often in a competitive bidding situation”. Especially, it seems, in the education area, institutions that have invested in their own facilities can be reluctant to share that investment with others. The fact that no-one mentioned out own institution in this respect is probably due more to politeness towards us than to a perception of the generosity of UEL’s institutional spirit – although we cannot definitively rule out the latter.

While several organisations are using the extranet successfully for connectivity purposes, not so many are using it to share applications. From all the interviews we conducted, the only concrete example of an extranet-hosted application which was mentioned was the delivery of an online learning package from the University of East London Business School to Newham Council Managers. Even with this package, awareness was both patchy and very imprecise. For instance, a relative insider commented: “There’s some link [from the Council] to UEL for online learning – I’m not sure how much it’s used. That’s the only application at the moment.” The impression we were usually given was that this was a completed rather than an ongoing project, but most people did not know its current status. Admittedly we were not talking to the people most centrally involved with the project, but the general lack of awareness and ignorance of detail can be taken to indicate that the task of ‘deploying applications across the extranet’ has not yet been very successfully carried out.

This may, however, change in the near future. Several people mentioned the ‘Creating Creativity’ project which aims to link the Three Mills Studios with post-production facilities in Soho and elsewhere: “Real applications will happen over that”. Likewise, the ‘Creative Industries Telematics Project’ aims to incorporate an online booking system that can be used by other organisations apart from its main owners, Stratford Circus and the Theatre Royal (although it is not clear how far this application will involve the extranet.)

The Newham Gateway

The Newham gateway is essentially a web site that aims to put together pointers to all the online resources either in or relating closely to Newham, and to provide some resources of its own (e.g. an online calendar of local events). The term ‘gateway’ adequately conveys this meaning, but to some extent it has been superseded by the term ‘portal’, which has become generally used to designate any site which – by drawing together a collection of applications and services – serves as a starting point for users to reach other services. The home pages of search engines/catalogues such as Yahoo! and AltaVista, as well as those of large Internet service providers like AOL and Freeserve, are well-known examples of portals. The Newham equivalent - available at http://www.newham.net - is intended to be a local rather than a worldwide, national or subject-based portal. In the following section we will use the original term ‘gateway’ to describe it, although where respondents have used ‘portal’ we have not changed this.

As with the extranet, we extracted a small number of tasks relating to the gateway from the original project proposal and asked our respondents how far they were aware of these tasks and, where appropriate, how far they thought the tasks had been accomplished. The list we showed was as follows:

1. Design the gateway/portal interface

2. Launch the gateway

3. Spread awareness of the gateway amongst Newham Internet users

4. Achieve high usage levels of the gateway

Although the first prototypes of the gateway had been accessible for some time, in the weeks immediately preceding our interviews the gateway had been redesigned and this had been extensively publicised on NeOn mailing lists. We were therefore not surprised that most of our respondents showed reasonably good awareness of its existence and that many had accessed it in the previous week or two. So, there was widespread awareness that the gateway had been designed, although opinions differed over whether it had been launched, which was almost certainly related to some confusion about whether the word ‘launched’ meant ‘made available on the Web to the general public’ (which it clearly was) or ‘opened with ceremony and publicised widely’ (which most people thought had not happened).

While (at the time of writing) we have yet to see advertisements for the gateway on the sides of buses or in the local media, respondents did make us aware of some efforts to publicise the gateway’s existence. The most notable place was on the home page of Newham Council’s web site, where a link to “search all Newham web sites at www.newham.net” is displayed in a small typeface but a prominent position. This is a start which can be built on once the site is deemed ready for wider publicity.

Most respondents commented favourably on the design of the gateway and thought it an improvement on the prototypes. One respondent with expert knowledge made a number of detailed technical criticisms, but these were not seen as making the site seriously difficult to use. A couple of respondents commented that other classes of people (for instance, ‘politicians’ or ‘media types’) found the site not stimulating enough visually, but this was in both cases immediately followed by a remark to the effect that ‘they don’t understand the usability issues’.

Given the short period of time in which the gateway has been available, and the lack of general publicity, it is probably too early to judge how successful it will be in achieving high usage levels. The usage figures for the last months of the year 2000 have been published on the NeOn ‘core’ mailing list. We have converted the cumulative figures to monthly figures – we hope we have done this correctly – and present a selection of the more meaningful statistics below. Given the variability of the number of ‘hits’ generated when users look at different pages, the most meaningful figure is probably that of the number of visits, while the number of separate visitors (or at least separate user addresses) is also interesting. The figures generally reflect the burst of activity around the gateway in October and November, followed by a tailing off in the run-up to Christmas.

The Newham gateway – statistics, August-December 2000

(source: Newham Online ‘core’ mailing list)

	Aug	Sep	Oct	Nov	Dec
Total hits	9,867	9,306	12,431	13,935	8,129
No. of visits	1,814	2,186	2,225	2,929	1,703
No. of visitors	829	1,007	740	1,179	607

Within the gateway, the most popular pages appear to be the home page, the calendar (which several respondents cited as being potentially useful) and the search results pages. While the vast majority of visitors were identified as coming from UK addresses, there were a small number of visitors (more than would be expected from automated programs, e.g. those generated by search engines) from Australia, Germany, New Zealand, Canada and the Netherlands. It is also possible that there were some visitors from the United States, but the way addresses are classified in the statistics does not allow them to be distinguished from UK people using ‘top level domain name’ addresses such as those ending in .com or .net. At a seminar at the University of East London early in 2000, one participant (from Manchester) identified a significant amount of usage of a locally-based web site by people who had moved away from the area, so there may be an opportunity to market the gateway (and the borough generally) to the ‘Newham diaspora’.

Most respondents who had accessed the gateway thought that it was a good thing, e.g. “The gateway project is excellent. … It’s a good site, with a good framework. … It’s got huge potential for development”; “People with network access would use this resource”. However, one sounded a note of caution. The gateway is most useful when people begin their Internet search by asking where certain facilities and services might be found. But this respondent thought that most people begin by asking what is it that they need and orienting their search towards that subject area. The respondent is associated with services that are relevant for people living both inside and outside the borough, and believes that a ‘what’ rather than a ‘where’ focus is the normal attitude of almost all organisations and individuals outside the Council: “People in London don’t care about local authority boundaries”. Another respondent noted – though not specifically in relation to the gateway – that commercial organisations were active in setting up subject-based portals in many areas, especially leisure services, and that it might be more useful for NeOn to concentrate its efforts in areas which were not so well covered by the private sector. Against that, it can be argued that a geographically-based gateway becomes more valuable as it increased the scope of the resources it brings together.

NeOn’s self-assessed strengths and weaknesses

One proven procedure in the field of evaluation, according to a colleague with an international reputation in the area, is the confrontation of an organisation’s or a project’s own analysis of its strengths and weaknesses with the perceptions of its partners, funders, suppliers or customers. In the Spring of 2000, the Newham Online steering group had organised a meeting where group members had discussed NeOn’s strengths, weaknesses, opportunities and threats – a so-called SWOT analysis – which was published on the NeOn web site. We were therefore able to make a selection of the (long) lists of identified strengths and weaknesses and ask our respondents to judge how far they agreed that these were indeed strengths or weaknesses and how important they believed them to be.

The list of self-identified strengths (which were not necessarily listed on the web site in order of importance) which we presented was as follows:

1. has a broad vision

2. provides a context in which to develop specific projects

3. has created a real community

4. has a growing profile, reputation and influence

5. encourages a spirit of sharing (e.g. provides information, lets people communicate, publicises local resources, etc.)

There was almost universal agreement that the first point was correct – that NeOn has a broad vision. It was also seen as one of NeOn’s great strengths. However, more than one respondent, in almost identical words, went on to say that, “A broad vision can be both a strength and a weakness”, meaning that a broad vision can become a weakness “if you can’t live up to it, if you spread yourself too thinly”. Two respondents linked this explicitly to one of the weaknesses (below): that of a danger of exaggerated perceptions. Most people also agreed that NeOn provides a context for specific projects, although a couple of respondents added that there weren’t yet enough of these.

There was less consensus about whether NeOn had created ‘a real community’. Some people clearly thought it had done this, while others were not so sure. The people who were most enthusiastic tended to be those who had been involved in the setting up of the infrastructure or were among the most active users of NeOn-related mailing lists. “NeOn has created a real community among the organisational people who belong to it”. Those who were more sceptical tended to be those people who were more concerned with bringing together local groups, who related more to content and applications than to network infrastructure and/or who were more distant from the ‘centre’ of NeOn (there is a significant amount of overlap in these categories). For instance: “Is there a real community? A community of what? There’s a strong partnership amongst ICT people, but that’s different from a community”. Yet others took a position somewhere in the middle: “There is a real community, but it’s far too small”; “Potentially, NeOn can create a real community”. One respondent who had taken this ‘middle’ position when asked about strengths later went on to be more sceptical: “Participants often mean themselves when they say ‘community’”, but this was a general point and was not elaborated as a specific criticism of NeOn.

Most people agreed that NeOn had a growing profile, reputation and influence (“They’ve put a lot of work in”, was one comment), though some people thought that this influence was still not big enough and others linked the growing reputation to the potential weakness of generating exaggerated expectations. Also, most respondents agreed that NeOn encouraged a sharing approach, although a couple of people expressed doubts over whether this encouragement had borne much fruit: “It encourages sharing, but it’s not clear how successful that is”.

As for weaknesses, the list we extracted and presented was as follows:

1. too few resources

2. skills base is too small

3. lack of co-ordination between key stakeholders

4. lack of support from some local institutions

5. exaggerated perceptions of NeOn’s capabilities (leading to lack of involvement by partners/users)

Just about everyone agreed strongly that NeOn suffered from having too few resources. The only voice of mild dissent was from someone who asked, “What level of resources might be too few?” However, this person admitted to very imprecise knowledge about the amount of resources available to the initiative. For the rest, it was clear that NeOn needed to be better resourced. “Yes, NeOn needs more resources”, “Resources are a big problem” were typical responses here. One interesting response hinted that this problem was the result of NeOn’s values and activities: “NeOn has resource problems because it has chosen to associate with community groups” (the context made it clear that this was instead of pursuing a commercial orientation). The most pointed comment was, “NeOn is stupidly underfunded. It’s a joke.” Further statements about NeOn’s lack of resources were linked to the discussion of its organisation and will be discussed below.

Interestingly, there was much greater disagreement over the issue of whether the skills base of NeOn was too small. Some people agreed, but others said things like “I don’t know if the skills base is too small – maybe”, or “I wouldn’t have said the skills base is too small – I know the quality of the people”. Those people who thought the skills base was indeed too small tended to be people who were working actively within NeOn – perhaps realising how much they were being stretched or wishing that they had more people to take on responsibilities: “Yes, there are too few activists”, said one. People who doubted whether there was a shortage of skills tended to be people who had been helped by people within NeOn: their comments reflected the perceived skills and qualities of the people who, in their eyes, were making NeOn effective. An appreciation of the skills of the key people within Newham Online was associated with the recognition of its growing reputation. The final word here goes to one respondent who linked the skills issue with that of NeOn’s organisation: “The narrow skills base is not a weakness but a development issue”.

Items three and four tended to be taken together. Respondents often said they were not in a position to judge, except for their own organisation (where they often identified lack of support at other – often high – levels within their organisation). A couple of interviewees, however, mentioned that some people might not want other areas of their organisation to become too supportive: “Support from local institutions is an interesting issue: some activists don’t want control taken out of their hands”. One person said their own organisation was “suspicious of outsiders”, and the comments made above in relation to the extranet, to the effect that some partners are in competition with each other and are therefore reluctant to share and co-operate, may also have some relevance here.

Finally, as indicated already, there was a significant amount of support for the idea that one of Newham Online’s weaknesses might be a tendency to foster exaggerated perceptions. This was linked to the breadth of NeOn’s vision, and in one case to trust in the skills and abilities of NeOn’s Director. One person, however, said they “don’t know who might have exaggerated perceptions”, and another – while admitting the existence of the weakness – was relatively unconcerned: “Yes, there are exaggerated perceptions, but NeOn has to talk up its impact to get money. The trick is not to believe your own hype.”

Reputation

We asked how Newham Online was perceived by people with whom the respondents came into contact. From their answers, NeOn’s reputation overall appears to be a positive one. For instance, “Other people, when they talk of NeOn, are generally very positive”; “There’s a lot of cynicism within local partnerships, so maybe negative comments don’t surface, but my colleagues are positive, so that’s a good recommendation”. NeOn’s reputation was, though, often divided into external and internal sectors, although the context of ‘external’ and ‘internal’ varied according to the respondent; it could mean inside and outside the borough, or the Council, or the core activist grouping. All agreed that NeOn had a positive reputation outside Newham. Inside, amongst partners and user groups, attitudes could be more ambivalent, e.g. the opinion that NeOn hasn’t presented itself as offering something to the community, and therefore ordinary people don’t know much about it; they are not necessarily hostile, but they do not quite know what exactly it does. Sometimes, people’s judgment of NeOn’s reputation depended on how close they saw themselves as being to it. Those who felt not so close tended to be more reserved or to talk in very general terms. This assessment was interpreted by one partner as related to the belief that IT is important, therefore NeOn “must be important”. Internally, for instance within the Council, people were seen to have confidence in it, but some respondents identified classes of people who had a lack of clarity about what it is (inevitably, perhaps, politicians were mentioned as one such class). For some outside the core activist grouping, NeOn’s reputation, at least in the early days, was that of a closed technical group, a “bunch of anoraks”.

As we asked about NeOn’s reputation, we became aware of another reputation that respondents were keen to talk about. A high proportion of the people we talked to, either explicitly in this context (questions about reputation) or elsewhere in the conversation, expressed a very positive opinion about the abilities and attitude of the Council’s IT Department. This is, of course, directly related to NeOn, as Newham Council IT Dept. is a key partner as a provider of resources and management functions, and an enabler of connectivity. The most common expressions were related to the Department’s perceived openness and flexibility, and its attitude towards its customers. One comment, from someone in a position to make comparisons between local authorities, was: “People are very positive about LBN’s IT Department. Other IT departments are focused on secure financial systems and are introspective. Newham’s IT people are ‘extraspective’; they have the idea of putting information out to the citizens”. People we spoke to within the Department tended to cite their colleagues as possessing these qualities, while people outside praised the team as a whole. The competence and attitude of the Council’s IT team was seen to be an important part of what some people called Newham’s ‘special configuration’, which will be discussed later when we consider responses to the question of how far the experience of Newham Online can be applied in other places. Also, some of the people who were cited as having skills that were relevant to NeOn’s reputation were members of the Council’s IT team. Although we do not wish to imply that the Department is beyond criticism, clearly it is doing some things very well.

Value for money

Finally in this section, we asked whether respondents considered that NeOn had up to now provided good value for money. For most respondents, NeOn was indeed considered to provide good value for money. One partner assessed it as providing very good value in its role as an enabler, focused on helping groups rather than doing things itself. Another partner highlighted the value of the extranet. Those people who were most positive tended to be either people who had been closely associated with the setting up or use of NeOn’s infrastructure and who had a good idea of commercial costs, or people who had been helped directly by NeOn members with advice or informal consultancy. There were no respondents who thought that, in general, NeOn had not provided a good return on investment. A couple of people, however, declined to pass an opinion, either because they felt they did not have enough detailed knowledge or they were unwilling to say without detailed financial information: “I would need to see the balance sheet before passing a verdict on whether NeOn has been value for money”. One respondent gave a more graduated answer, distinguishing between value at the local level (“Good value for local funders – it spreads Newham’s reputation”), at national level (“There hasn’t been much national funding – just a bit of SRB – so it hasn’t mattered if NeOn is good value”) and at European level (“At European level, maybe it isn’t good value. NeOn works well locally, but it isn’t ‘clonable’ in European terms.”). This last issue, of how easily the experience of NeOn can be transplanted, will be taken up again in the final section.

3. Newham Online’s Organisational Structure

The response of many of our respondents, when asked to describe Newham Online’s organisational structure, was a pause, followed by a statement which either took the following form: “It’s hard to describe”; “It’s indescribable”, or focused on the Director: “The organisational structure of NeOn is Richard Stubbs”. This was then often followed by a description of NeOn as an informal organisation: “NeOn is very unconstituted – this is very unusual”; “NeOn is an act of will”; “The organisation of NeOn is informal” and some mention of the existence of the steering group. Emerging from all this was a broad consensus about what NeOn’s structure(-lessness) actually is – where it was known at all – but there was no such consensus on how this was to be evaluated.

The extent to which respondents felt able to describe Newham Online’s organisational structure varied considerably. For instance, one person said, “I have only a vague understanding of NeOn’s organisation. It’s a partnership, I believe it’s formally constituted, and it has a trading arm… I imagine NeOn has employed staff.” Another admitted, “I’m not totally sure what the organisation is.” Often, people mentioned specific individuals or organisations which they believed to be connected somehow with NeOn but they were not sure how the connections worked exactly. For instance, “I’m not sure how Mike and Computer Access fit in, or how Michael Mulquin fits in.” Or (from a different interviewee), “There’s also Go2Find. I’m not sure how that fits in.” There was a group of respondents who felt they had a fairly good idea of how NeOn is organised, and another group – more distant from the centre of NeOn’s activities – who were either less sure or less inclined to trust their own perceptions.

The ‘broad consensus’ mentioned above consists of three basic features of Newham Online’s organisation. It is informal, Richard Stubbs is the central figure, and the steering group plays a role. In the fol

Newham Online: Evaluation and Outlook

University of East London