Dear all

I think the correspondence tables have not been discussed. What would need to be done to meet the objectives of the hackathon?

best,

Miguel (F. Astudillo)

I think the correspondence tables should just be converted to RDF for entry into the database. In my understanding, there is nothing a correspondence table does that an RDF database does not also do.

The necessary tasks would be to identify the appropriate predicates to describe the relationships. This crosses into the #ontology discussion, and is outside my expertise anyway, but the #skos ontology seems to have some of the right terms in its mappingRelations superclass For instance: skos:exactMatch and skos:closeMatch seem like good candidates.

Again I am outside my direct experience, but in the example table here: https://github.com/BONSAMURAIS/bonsai/blob/da0f7d82161b8a296a0b571d3c2fa2f1a137b8ca/FAO_vs_EXIOBASE_prod_classification_correspondence.xlsx
I would do something like the following, to encode the first row of the table:
Note the difference between the "rdf" vocabulary and the "rdfs" vocabulary.

rdf = https://www.w3.org/2001/sw/RDFCore/Schema/200203/
rdfs = https://www.w3.org/TR/rdf-schema/#ch_introduction
(default) = https://uri.for.bonsai.namespace

 - Within the BONSAI namespace, create new signifiers for the collections between which the correspondence is made, like

FAO_products rdf:type skos:Collection
EXIOBASE_products rdf:type skos:Collection

 - Then for each row in the example, create a signifier and assign its label

FAO_Agave_fibres_nes skos:member FAO_products
FAO_Agave_fibres_nes rdfs:label "Agave fibres nes"

 - Similarly for each column:

EXIOBASE_plant_based_fibres skos:member EXIOBASE_products
EXIOBASE_plant_based_fibres rdfs:label "Plant-based fibres"

 - Then add the relationship

FAO_Agave_fibres_nes skos:closeMatch EXIOBASE_plant_based_fibres

Somebody correct me if I'm off base, but I think this is how we move from XLS to RDF.

Michele is supposed to get this organized, but I can provide some inputs from my side:

This working group should have multiple outputs which build upon each other.

a. Define a standard for correspondence tables, and convert everything we can find to that format.

I am 100% convinced that this format should be https://frictionlessdata.io/specs/data-package/. Each correspondence table would consist of a CSV with the raw data, and a JSON file with the metadata. Our task would be to define the metadata format (building on what the OKFN has already done, we just need to fill some things in). The idea is that the metadata can be consistent and therefore machine-readable.

For the CSV, we should discuss. 1-1 correspondence is easy. I think that 1-N and N-1 is also easy; one could have a two column format:

    foo, bar1
    foo, bar2

and

    foo1, bar
    foo2, bar

We could also have a third column that would give weights when more than one mapping is possible.

But we want to do this right, so should look at the various proposal defined for "crosswalk" tables, how these mappings are stored in open source LCA software, etc.

This should be a new repo, with one directory for the final product, one directory for the jupyter notebooks/whatever used to convert the raw data, and a third directory for the input data in its "native" form (if applicable). See https://open-power-system-data.org/ for inspiration.

I see that Brandon has just responded to this question with a totally different answer, so I look forward to a good discussion! I believe that data packages are language and community agnostic and are therefore much more of a community resource than something RDF specific would be. As always, the more value we provide to our information, the higher the chance that it is used by others, and then maintained by others :)

On the other hand, Brandon's approach allows us to express relationships much more concretely, and we would need this level of detail at some point in any case.

b. Set up a simple web app at correspondence.bonsai.uno that would return these correspondence in multiple formats.

Technically quite easy, and would be a good exercise to set up a BONSAI python web app skeleton.

c. Write a Python library that would allow the easy application of these correspondence tables.

During and maybe after the hackathon (or not - surprise me :)

Thank you Miguel for starting this hashtag and thanks Chris and Brandon for the inputs.

This discussion seems the right place to kick start the Group on Correspondence tables. 

I have just created a new directory on GitHub for the work of this sub-group. The link to this discussion is mentioned in the Readme file.

I am currently working on the deliverable I listed under "Before the Hackathon". You will see the updates in the folder soon.

Suggestions and contributions are as always very welcome :) 

Mic

After some reflection, Brandon's approach is much more sensible than mine.

We would then need to either generate a preliminary vocabulary
ourselves, or get one from the #ontology folks.

Do we know all possible verbs we want to include? For example, in 1-N
mappings do we have a verb for the "default" option?

--
############################
Chris Mutel
Technology Assessment Group, LEA
Paul Scherrer Institut
OHSA D22
5232 Villigen PSI
Switzerland
http://chris.mutel.org
Telefon: +41 56 310 5787
############################

We have a good start! I added a checklist for pre-hackathon release, and added that to the hackathon planning board.

Michele, please add a short working group description to the hackathon README.

@Chris Done. Is there anyone that subscribed to this group that is not listed as a group member right now?

@All, I am working on collecting under our group repo all the info/material/guidelines we have already found/developed on this topic. Get in touch with me if you have material/ideas

Should we schedule a call sometime early next week (e.g. Monday afternoon 5pm CET) ?

Working group self-assignment is here:
https://bonsai.groups.io/g/hackathon2019/responses?pollid=4642

--
############################
Chris Mutel
Technology Assessment Group, LEA
Paul Scherrer Institut
OHSA D22
5232 Villigen PSI
Switzerland
http://chris.mutel.org
Telefon: +41 56 310 5787
############################

Hi Michele,

I selected to work in this group. How can I help now?

I am also available to have a call Monday.

--
Tiago G. Morais
Instituto Superior Técnico - Universidade de Lisboa/University of Lisbon
MARETEC/LARSYS
Área Científica de Ambiente e Energia/Environment and Energy Scientific Area
Departamento de Engenharia Mecânica/Department of Mechanical Engineering
Pavilhão de Mecânica I | Av. Rovisco Pais, 1 | 1049-001 Lisbon, Portugal
Tel: +351 21 841 98 77 | ext. 3877

1. See if there are any differences in the elementary flows lists of
exiobase version 2 and 3 (even a single character difference is enough
to need a mapping).

2. Map ecoinvent elementary flow list (from
https://github.com/BONSAMURAIS/Correspondence-tables/blob/master/EcoinventElflow3.2_to_ILCD.csv)
to EXIOBASE elementary flow list

3. Map exiobase v3 elementary flow list to US EPA elementary flow list
(https://github.com/USEPA/Federal-LCA-Commons-Elementary-Flow-List/blob/master/fedelemflowlist/output/FedElemFlowList_0.2.csv)

On Fri, 15 Mar 2019 at 11:35, Tiago Morais
<tiago.g.morais@tecnico.ulisboa.pt> wrote:

Hi Michele,

I selected to work in this group. How can I help now?

I am also available to have a call Monday.

--
Tiago G. Morais
Instituto Superior Técnico - Universidade de Lisboa/University of Lisbon
MARETEC/LARSYS
Área Científica de Ambiente e Energia/Environment and Energy Scientific Area
Departamento de Engenharia Mecânica/Department of Mechanical Engineering
Pavilhão de Mecânica I | Av. Rovisco Pais, 1 | 1049-001 Lisbon, Portugal
Tel: +351 21 841 98 77 | ext. 3877


-----Original Message-----
From: hackathon2019@bonsai.groups.io <hackathon2019@bonsai.groups.io> On Behalf Of Chris Mutel
Sent: 15 de março de 2019 10:27
To: hackathon2019@bonsai.groups.io
Subject: Re: [hackathon2019] #correspondencetables - what needs to be done?

Working group self-assignment is here:
https://bonsai.groups.io/g/hackathon2019/responses?pollid=4642

On Fri, 15 Mar 2019 at 11:20, <michele.derosa@bonsai.uno> wrote:

@Chris Done. Is there anyone that subscribed to this group that is not listed as a group member right now?

@All, I am working on collecting under our group repo all the info/material/guidelines we have already found/developed on this topic. Get in touch with me if you have material/ideas

Should we schedule a call sometime early next week (e.g. Monday afternoon 5pm CET) ?

--
############################
Chris Mutel
Technology Assessment Group, LEA
Paul Scherrer Institut
OHSA D22
5232 Villigen PSI
Switzerland
http://chris.mutel.org
Telefon: +41 56 310 5787
############################

--
############################
Chris Mutel
Technology Assessment Group, LEA
Paul Scherrer Institut
OHSA D22
5232 Villigen PSI
Switzerland
http://chris.mutel.org
Telefon: +41 56 310 5787
############################

I will start working on the point 1.

--
Tiago G. Morais
Instituto Superior Técnico - Universidade de Lisboa/University of Lisbon
MARETEC/LARSYS
Área Científica de Ambiente e Energia/Environment and Energy Scientific Area
Departamento de Engenharia Mecânica/Department of Mechanical Engineering
Pavilhão de Mecânica I | Av. Rovisco Pais, 1 | 1049-001 Lisbon, Portugal
Tel: +351 21 841 98 77 | ext. 3877

Hi Michele,

Like Tiago, I also opted for this group to offer my help.
I added my name to the readme on github and created a pull request.

I will start to work on point 2 from Chris' list.

Hi all,

 

I already finished the correspondence between v3 and v2 of exiobase, but I’m not authorize to upload in the github. Thus, I attached the file year.

 

Meanwhile, I will also start to work on the point 3 from Chris’s list.

Great! What's your GitHub usr? I'll add you. Please update the overview file too (see readme in repo)

It is tgmorais1.

Ok ok!

Hi all,

I just want to let you know that the latest version of Exiobase multi-regional hybrid tables, i.e. v3.3.17, is on exiobase.eu

I have changed the labels of the HIOT product classification in order to improve the consistency with HSUTs. I will soon upload the correspondence table on Github. Unless big issues, the final format should be that one. Then, if you want, I can add other data.

Best,

Stefano

Hi all,

 

I already finished the correspondence between v3 and v2 of exiobase, but I’m not authorize to upload in the github. Thus, I attached the file year.

 

Meanwhile, I will also start to work on the point 3 from Chris’s list.

-- 
Best,
S.

Hi,

I just uploaded the correspondence table between EXIOBASE and US EPA.
Nevertheless, there are flows in the EXIOBASE that don't have correspondence in US EPA flows (they just a few cases)

Cheers
Tiago

Hi,

I started on the correspondence between the elementary flows of ecoinvent and exiobase. Is anyone aware of any classification scheme for the exiobase resources and emissions?
Or should they be manually mapped on the names?

@Stefano: are there such concordances available for exiobase? I got a bunch of concordances from Konstantin Stadler but non on the environmental flows.

@Tiago: How did you go about this?

Thanks,
Arthur

Hi Jacob,

I have this old file I used to convert Exiobase HSUTs extension into a Simapro format.

Best,

Stefano

Hi,

I started on the correspondence between the elementary flows of ecoinvent and exiobase. Is anyone aware of any classification scheme for the exiobase resources and emissions?
Or should they be manually mapped on the names?

@Stefano: are there such concordances available for exiobase? I got a bunch of concordances from Konstantin Stadler but non on the environmental flows.

@Tiago: How did you go about this?

Thanks,
Arthur

-- 
Best,
S.

Hi all,

I just checked the correspondence between Exiobase v2 and Nace Rev.2 and I've noticed that some sectors of the Nace Rev.2 were not included in the table. This means that the table was not exhaustive. Is there any reason for that?

However, I have uploaded a new version with suffix v2 but I did not check all the sectors. I think we will need some cross-check later on.

Best,

Stefano

Hi Jacob,

I have this old file I used to convert Exiobase HSUTs extension into a Simapro format.

Best,

Stefano

On 20/03/2019 17:04, arthur.jakobs@... wrote:

Hi,

I started on the correspondence between the elementary flows of ecoinvent and exiobase. Is anyone aware of any classification scheme for the exiobase resources and emissions?
Or should they be manually mapped on the names?

@Stefano: are there such concordances available for exiobase? I got a bunch of concordances from Konstantin Stadler but non on the environmental flows.

@Tiago: How did you go about this?

Thanks,
Arthur

-- 
Best,
S.

-- 
Best,
S.