Skip to main content
Bookmark and Share

K. Data Sharing & Data Sharing Protocols

Dataset sharing and development of data sharing protocols are critical initiatives if a local area is to gain a broader, more holistic understanding of the individual, the barriers they face to employment and how provision can be improved to tackle these issues. It is also required, to some extent, simply to get a system to work as intended. 

In the context of an Employability Management Information System there are several technical and non-technical considerations:

1. Non-Technical Aspects of Sharing Data

Data sharing can occur between organisations in an existing Partnership or indeed between organisations where no formal agreed contract or partnership constitution exists but there are a set of common aims or objectives from sharing any data.

An existing Partnership usually (but not always) has a legal and constitutional agreement in place which negates (to some extent) the need for any additional set of data sharing protocals or guidance. For example, an employability partnership may have an existing basis to insist that its partners use a new system, migrate their data or integrate their existing systems with any new system.

1.1 Establish a basis for Sharing Data

Before any two organisations can reach a point of drafting a data sharing protocal, it is important to focus on the basis for sharing. The purpose of Data Sharing, in an Employability context, is about management information and the value from shared information (rather than the technical or legal barriers that may be apparent).

Data sharing provides an increased understanding of where people go, when they engage and a more holistic view of the barriers they face. It is a critical activity if management information is truly valued. Data sharing can be encouraged through active cross organisation participation in Management Information Outputs sessions. Data Sharing is an ongoing process that may culminate in data integration but is more likely to result in increased learning and knowledge sharing.

Practitioners are more aware of the value of shared data. Discussing data sharing at practitioner level and comparing datasets without necessarily disclosing 'confidential' information should be the first steps. Data sharing initiatives that solve organisational or legal barriers (DPA) or challenge organisational or legal constraints should also be encouraged. It is important to seek to identify the value of your data for others.    

By commencing this activity with discussions around management information outputs and activities such as comparison of datasets (anonymously if need be), the value of shared data is quickly understood and relationships start informally at practitioner level and develop and strengthen upwards as necessary. Extending the informal relationship into the creation of a data sharing protocal (as indicated previously) only helps support this relationship longer-term and ensures the data is still valued even when the original staff members move on. This practice also provides definitive statements on the data protection issues surrounding any initiative, combating speculation. See link to the 'Framework code of practice for sharing personal information' from the Information Commissioners Office.

Data sharing is often seen as a senior management issue or a technology or system issue. This approach focuses on how management information should be shared rather than exploring the value from shared information. Without data value being discussed, data sharing appears like a risky venture, where data security, data protection and system integrity are all potential losers. Data sharing will not happen quickly at senior management levels unless there is ground level support between organisations.

1.2 Create a Data Sharing Protocal

Partners that are less closely aligned need to create Data Sharing Protocals. Simply put, these are agreements or memoranda of understanding (sometimes legally enforceable) that clearly define the basis of the agreement. A data sharing protocal therefore, will do the following:

  • Define the high level objectives (from the initiative)
  • Describe the initiative itself
  • Detail the actors involved and their respective roles and responsibilities
  • Detail non-technical data sharing processes and practices
  • Detail or reference data standards and/or common datasets
  • Identify the core elements of the technical solution (referencing analysis, shared processes, technical processes and frameworks, and any operational timescales)
  • Identify any foreseeable risks and how these will be managed. Including issues such as data protection, confidentiality, security etc.
  • Provide rules of engagement and indicate SLAs (Service Level Agreements). Consider, at what point does an agreement begin to dissolve. What actions are necessary to recognise issues and maintain stability. Do the actors have enough influence to ensure the long-term stability of any agreement.

2. Technical Aspects of Sharing Data

There are two main types of IT system data sharing: System Migration and System Integration.

2.1 System Migration

System migration is the translation of system data from one format to another or from one database to another. This is usually an automated process conducted by the appointed system supplier (in conjunction with the project team) and involves the following steps:

  • Analysis and design: data from the old system is mapped to the new system, translating old formats and structures to new.
  • Extraction and data loading: data is extracted (through appropriate software scripts) and then written (loaded) on the new system
  • Validation and repair of data: data is checked for data accuracy and process validation (i.e. it supports new processes). Where appropriate the data is cleansed or repaired. This process can occur a number of times.
  • Implement: use of data in the new program or system

The scope of work is usually identified within the Project Brief and defined more fully post-implementation and following analysis by the system supplier or an IT specialist. System migration is typically a one-off process.   

2.2 System Integration

System integration in this context, involves creating a perpetual flow of information between two discrete systems. Usually, this involves one system being able to write specified information to another on an ongoing, pre-determined basis. It can involve daily updates between systems or as and when new data is captured from one or other system(s). Integrating systems require use of appropriate technologies, such as XML-based schemas or the creation of web services, and the involvement of technology experts.

Before creating the ongoing technical solution, the system supplier or IT specialist will conduct the following:

  • Analysis and design: data from the source systems are mapped, translating old formats and structures to a target dataset and data flow processes defined (i.e. the business rules by which information will flow between two systems is defined).
  • Integration tool development: This involves creation of a suitable XML schema, web service and/or related scripts in order to extract, load, write and overwrite specific data on a continual basis
  • Extraction and data loading: data is extracted and then written (loaded) onto the target system or test system
  • Validation and repair: the schema or web service is checked for accuracy. Data is also checked for data accuracy and process validation. Where appropriate the integration tool or source data is revised and rechecked. This process can occur a number of times.
  • Publish: the web service or xml feeds are published with appropriate guidance for more general use

The scope of any System Integration work is usually identified within the Project Brief and defined more fully post-implementation and following analysis by the system supplier or an IT specialist. System integration is a perpetual process.

Our partners, in particular SDS and a local college, have been involved at the earliest stages of our MIS development. This has helped us make real progress in integrating our system with their mission-critical systems.

Respondent, Area E