Multiple wikipedia info boxes merged into one dbpedia resource

[mubashar1199]

In the templatemapping.scala file in dbpedia extraction framework, if wiki page have more than one infoboxes, they are merged into single dbpedia resource. Due to this alot of inconsistent data is populated in dbpedia for example:

1. Person having more than one birthDate.
2. Properties of one person in wikipedia is assigned to other person in the dbpedia.

Please see the wikipedia and dbpedia resource Helene_Demuth or Karen_McCarron as an example. In dbpedia Karen is mentioned as self parent due to merging of infoboxes. Several such problems exists due to merging of infoboxes.

I am working on this issue to remove it, Please assign me to this issue.

Thanks

[JJ-Author]

Do you have any idea how to approach this? I do not really have a good one except for filtering the infobox by its label (comparing with page title and ignoring the incorrect infobox). I would suggest to start with an analysis how many entities are actually affected.

[jlareck]

Also, there is a discussion in issues related to this problem I guess. Maybe you can check it: #17 . It seems to me that there was discussed almost the same problem

[mubashar1199]

Do you have any idea how to approach this? I do not really have a good one except for filtering the infobox by its label (comparing with page title and ignoring the incorrect infobox). I would suggest to start with an analysis how many entities are actually affected.

The idea is to check the name property of info box and if the name value doesn't matches with the page title then we save it as a new resource, also we check for label info box in the node title. I have made the required changes and now if the page have more than one info boxes and if any info box belongs to different entity and not the same entity as page title then it will be stored as a new resource in dbpedia. Please check the code and outcome of it before and after making changes.

//code

//before changes

//after changes

[mubashar1199]

Also, there is a discussion in issues related to this problem I guess. Maybe you can check it: #17 . It seems to me that there was discussed almost the same problem

Yes, the issue is very much similar. Their primary target was multiple templates about same entity, The solution have some problems as when we see the mentioned example enwiki:List_of_Playboy_Playmates_of_2012 in dbpedia, all the entities are still merged under one resource which was not intended.

[mubashar1199]

Do you have any idea how to approach this? I do not really have a good one except for filtering the infobox by its label (comparing with page title and ignoring the incorrect infobox). I would suggest to start with an analysis how many entities are actually affected.

About analysis of entities which are affected, i have run the sample query to get the rough idea of count of entities which are affected due to this issue. I ran the query to get list of Person entities who have more than one birthDate which returns hundreds of results. Many other inconsistencies also exists due to this issue. For-example when one person properties are assigned to other person, these might not be detectable by using queries but the data is incorrect.

[JJ-Author]

looks good to me. go ahead an create an PR please when you are ready. @jlareck please give assistance to write new tests according to the new test strategy.

[jlareck]

I think you, @mubashar1199, can try to add some pages that have multiple infoboxes to the minidump (here is the guide for how to add them: http://dev.dbpedia.org/Testing_on_Minidumps ) and write SHACL tests for extracted triples from those infoboxes (similar test: https://github.com/dbpedia/extraction-framework/blob/master/dump/src/test/resources/shacl-tests/instances/angela_merkel.ttl)

[mubashar1199]

I have added some pages in the minidump but when i run the tests, it is giving file not found exception, how to overcome this?

[jlareck]

@mubashar1199 There is something wrong with path. The path /home/mubashar/extraction-framework/dump/target/test-classes/minidumps/http:/wiki.xml.bz2 should not contain http:, but it must contain some language instead. So please check the /dump/src/test/resources/minidumps folder and check if there is no http folder, if it exists then remove it. Also, check the /en/wiki.xml.bz2 (if the pages, that you added, are from english wikipedia) to be sure that it contains your new pages (you can unzip it and in some text editor check if pages were added).

[jlareck]

Also I think the problem is in adding pages to /dump/src/test/bash/uris.lst. So, please send the screanshot of your new pages in that list.

[mubashar1199]

I deleted the http: folder and then new error arises of folder wikidata.org, also deleted this folder,. After this test are executing successfully but another error is being thrown due to which few tests are failed. Please see the screenshot of error and uri.lst
//Error

//uri.lst

[mubashar1199]

last 9 entries of uri.lst are newly added, and relative data of these pages is also downloaded to respective bz2 files

[jlareck]

Okay, so I guess all uris must start with https: not http: and that's why you had that error in the first case (Judith_Gould uri is that one that may produse the http folder). But the last one IRI Coverage Tests is interesting. Have you already tried to delete target folder in dump directory and run mvn install one more time in extraction-framework folder? If no, try it, maybe the problem is that something was not updated