[Medin_dacwg] Data Accelerator storage costs

Fri Nov 20 12:29:12 GMT 2015

Hi Ulric,

Not as far as I'm aware, the discussion with Pete wasn't in the context of the Data Accelerator.

However as I understand it, reaching the volume of data "needed" to meet Data Accelerator promise is entirely dependent on the granularity of each dataset, and that level of granularity hasn't been defined.
We could therefore treat each snippet or still as a dataset and very quickly reach the magic 8,000! I'm not seriously suggesting this as an approach, but it does highlight the lack of clarity around this activity. The only requirement on 'open-ness' that I'm aware of is that it has to be on data.gov.uk.

Cheers
Dan

From: Ulric.Wilson at jncc.gov.uk [mailto:Ulric.Wilson at jncc.gov.uk]
Sent: 20 November 2015 12:03
To: Dan Lear; Postlethwaite, Clare
Cc: medin_dacwg at mailman.nerc-liv.ac.uk
Subject: RE: Data Accelerator storage costs

Hi Dan,

Thanks for the info.  Do you know if there's been any discussion with Defra about this approach - is this approach 'open data enough'?

Ulric

Dr Ulric Wilson
Technical Project Manager
Joint Nature Conservation Committee, Monkstone House, City Road, Peterborough PE1 1JY
Tel: 01733 866853 Fax: 01733 555948
jncc.defra.gov.uk<http://www.jncc.gov.uk>

From: Dan Lear [mailto:dble at MBA.ac.uk]
Sent: 19 November 2015 20:25
To: Postlethwaite, Clare; Ulric Wilson
Cc: medin_dacwg at mailman.nerc-liv.ac.uk<mailto:medin_dacwg at mailman.nerc-liv.ac.uk>
Subject: RE: Data Accelerator storage costs

Hi Ulric,

I had this discussion with Pete Walker (NE) last week relating to MCZ video data.
The way we plan to handle it is as Clare describes, either an "edited highlights" video, ie edited by the data provider to contain a representative range of the habitats/species present in the video or as a series of stills that would work in the same way.
The original files would still be lodged with us as a DAC, and referenced in the metadata but would  be in offline,  cheap(er) storage and available on request.

Cheers
Dan

From: medin_dacwg-bounces at mailman.nerc-liv.ac.uk<mailto:medin_dacwg-bounces at mailman.nerc-liv.ac.uk> [mailto:medin_dacwg-bounces at mailman.nerc-liv.ac.uk] On Behalf Of Postlethwaite, Clare
Sent: Wednesday, November 18, 2015 9:55 AM
To: Ulric.Wilson at jncc.gov.uk<mailto:Ulric.Wilson at jncc.gov.uk>
Cc: medin_dacwg at mailman.nerc-liv.ac.uk<mailto:medin_dacwg at mailman.nerc-liv.ac.uk>
Subject: Re: [Medin_dacwg] Data Accelerator storage costs

Hi Ulric,
No MEDIN hasn't looked at storage costs in terms of the data accelerator project. As you know the MEDIN funding model is that the data funder pays to archive the data at a MEDIN DAC and the core funding capability of the DAC covers the storage etc (for the data that each DAC specialises in).

In terms of large files, UKHO are already serving their very large multibeam datasets so that should be ok (but UKHO would need sufficient warning to anticipate any significant numbers of datasets landing on their desks).

I believe the way that some of the DACs deal with serving large video files is to make a short section available for download so users can see if it is something that they are interested in and then the full dataset is available on request.

I'm copying  the DAC working group in case they have further comments on online access to large datasets like those you describe.
Best wishes,
Clare

From: Ulric.Wilson at jncc.gov.uk<mailto:Ulric.Wilson at jncc.gov.uk> [mailto:Ulric.Wilson at jncc.gov.uk]
Sent: 17 November 2015 09:08
To: Postlethwaite, Clare
Subject: Data Accelerator storage costs

Hi Clare,

As part of the MEDIN response to the data accelerator  / open data process has MEDIN looked at storage costs / types?

Obviously the ideal is for data to be accessible on a direct link - online storage - which is relatively expensive, especially if the data is video or multibeam and very large.

For large data types, how are MEDIN proposing to provide access and does Data Accelerator increase costs beyond the DAC archiving costs already in place?

Any thoughts welcome, as we're looking at all the MCZ video data and wondering if we _really_ need to have online storage for it (especially as many users would be put off by the file sizes anyway).

Ulric

Dr Ulric Wilson
Technical Project Manager
Joint Nature Conservation Committee, Monkstone House, City Road, Peterborough PE1 1JY
Tel: 01733 866853 Fax: 01733 555948
jncc.defra.gov.uk<http://www.jncc.gov.uk>

_____________________________________________________________________
This email and any attachments, is intended for the named recipient(s) only. If you are not the named recipient then any copying, distribution, storage or other use of the information contained in them is strictly prohibited. In this case, please inform the sender straight away then destroy the email and any linked files.

JNCC may have to make this message, and any reply to it, public if asked to under the Freedom of Information Act, Data Protection Act or for litigation. If you have a Freedom of Information/Environmental Information request please refer to our website page.

This message has been checked for all known viruses by JNCC through the MessageLabs Virus Control Centre however we can accept no responsibility once it has left our systems. The recipient should check any attachment before opening it.

JNCC Support Co. registered in England and Wales, Company No. 05380206. Registered Office: Monkstone House, City Road, Peterborough, Cambridgeshire PE1 1JY. http://jncc.defra.gov.uk/
________________________________
This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system.
________________________________

_____________________________________________________________________
This email and any attachments, is intended for the named recipient(s) only. If you are not the named recipient then any copying, distribution, storage or other use of the information contained in them is strictly prohibited. In this case, please inform the sender straight away then destroy the email and any linked files.

JNCC may have to make this message, and any reply to it, public if asked to under the Freedom of Information Act, Data Protection Act or for litigation. If you have a Freedom of Information/Environmental Information request please refer to our website page.

This message has been checked for all known viruses by JNCC through the MessageLabs Virus Control Centre however we can accept no responsibility once it has left our systems. The recipient should check any attachment before opening it.

JNCC Support Co. registered in England and Wales, Company No. 05380206. Registered Office: Monkstone House, City Road, Peterborough, Cambridgeshire PE1 1JY. http://jncc.defra.gov.uk/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.nerc-liv.ac.uk/pipermail/medin_dacwg/attachments/20151120/5a25fa0d/attachment-0001.html 