Discussion:
[Exist-open] JSON-LD Serializer for XQuery -> RESTXQ
Christopher Johnson
2017-04-20 08:53:09 UTC
Permalink
Hi list,

I am considering how to serialize JSON-LD using RESTXQ and am wondering if
anyone has implemented this already. I guess that this would first involve
extending the XQuerySerializer in eXist with a JSON-LD library.

My main use case is that I am currently using RESTXQ to serialize JSON, but
I would like to persist the xml namespaces for the elements in an @context.

Cheers,
Christopher Johnson
Joe Wicentowski
2017-04-20 11:54:17 UTC
Permalink
Hi Christopher,

You might take a look at https://github.com/baskaufs/guid-o-matic

Joe
Post by Christopher Johnson
Hi list,
I am considering how to serialize JSON-LD using RESTXQ and am wondering if
anyone has implemented this already. I guess that this would first involve
extending the XQuerySerializer in eXist with a JSON-LD library.
My main use case is that I am currently using RESTXQ to serialize JSON,
but I would like to persist the xml namespaces for the elements in an
@context.
Cheers,
Christopher Johnson
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Exist-open mailing list
https://lists.sourceforge.net/lists/listinfo/exist-open
--
Sent from my iPhone
Adam Retter
2017-04-20 13:39:18 UTC
Permalink
You could write your own serialization in XQuery and use the existing
text serializer, or as you say you would need to write your own
serializer in Java, the class to extend is
org.exist.util.serializer.XMLWriter and add an entry for it to
org.exist.util.serializer.AbstractSerializer
Post by Christopher Johnson
Hi list,
I am considering how to serialize JSON-LD using RESTXQ and am wondering if
anyone has implemented this already. I guess that this would first involve
extending the XQuerySerializer in eXist with a JSON-LD library.
My main use case is that I am currently using RESTXQ to serialize JSON, but
Cheers,
Christopher Johnson
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Exist-open mailing list
https://lists.sourceforge.net/lists/listinfo/exist-open
--
Adam Retter

eXist Developer
{ United Kingdom }
***@exist-db.org
irc://irc.freenode.net/existdb
Christopher Johnson
2017-04-21 11:40:33 UTC
Permalink
Hi Adam,

Thanks for the tip. Serializing JSON-LD from an non-RDF/XML nodeset would
be a whole lot easier if the org.xml.sax.XMLReader would simply allow
elements with an "@". If that were possible, then one could create
elements like <@context/>, <@id> and <@type> in the XQuery output and the
existing LD agnostic JSON serializer could be used (because JSON-LD is
always valid JSON).

I guess one way forward would be to first rewrite my local nodesets as
RDF/XML with XQuery (very annoying!) and then post-process RDFS validated
results with https://github.com/jsonld-java/jsonld-java-rdf2go in a custom
serializer. Adding an annotation that specifies output method
"application/ld+json" to RESTXQ would then work only on valid RDF/XML.

Cheers,
Christopher
Post by Adam Retter
You could write your own serialization in XQuery and use the existing
text serializer, or as you say you would need to write your own
serializer in Java, the class to extend is
org.exist.util.serializer.XMLWriter and add an entry for it to
org.exist.util.serializer.AbstractSerializer
Post by Christopher Johnson
Hi list,
I am considering how to serialize JSON-LD using RESTXQ and am wondering
if
Post by Christopher Johnson
anyone has implemented this already. I guess that this would first
involve
Post by Christopher Johnson
extending the XQuerySerializer in eXist with a JSON-LD library.
My main use case is that I am currently using RESTXQ to serialize JSON,
but
Post by Christopher Johnson
I would like to persist the xml namespaces for the elements in an
@context.
Post by Christopher Johnson
Cheers,
Christopher Johnson
------------------------------------------------------------
------------------
Post by Christopher Johnson
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Exist-open mailing list
https://lists.sourceforge.net/lists/listinfo/exist-open
--
Adam Retter
eXist Developer
{ United Kingdom }
irc://irc.freenode.net/existdb
Adam Retter
2017-04-21 18:20:22 UTC
Permalink
Post by Christopher Johnson
Thanks for the tip. Serializing JSON-LD from an non-RDF/XML nodeset would
be a whole lot easier if the org.xml.sax.XMLReader would simply allow
existing LD agnostic JSON serializer could be used (because JSON-LD is
always valid JSON).
Well <@cotext/> etc is not valid XML, hence why the XMLReader won't
let you do it.

However if you know your mapping from XML to JSON-LD then you could
implement something at the same level as
org.exist.util.serializer.TEXTWriter to output your JSON-LD.
--
Adam Retter

eXist Developer
{ United Kingdom }
***@exist-db.org
irc://irc.freenode.net/existdb
Christopher Johnson
2017-04-22 06:00:39 UTC
Permalink
Sure, it is possible to write JSON-LD as text, but this is not a proper
serialization of JSON. There is valiid JSON and then there is plain text,
and they are not the same. It seems also not practical to pretend that
JSON is text for the benefit of XQuery or XML. For example, reading JSON
as unparsed text into a nodeset using XSLT, and then sending the output to
json:contents-to-json will not yield valid JSON (at least for me). I
should qualify that my interest is not in hacking JSON-LD to make it
function within the functional limitations of XML (and the eXist db), but
rather in providing a valid JSON-LD serialization (and RESTXQ) output
method for *any* RDF/XML that could be stored in an eXist db collection. I
still suspect that this could be a relatively common use case that is
supported by already well-developed libraries.

Cheers,
Christopher
Post by Adam Retter
Post by Christopher Johnson
Thanks for the tip. Serializing JSON-LD from an non-RDF/XML nodeset
would
Post by Christopher Johnson
be a whole lot easier if the org.xml.sax.XMLReader would simply allow
existing LD agnostic JSON serializer could be used (because JSON-LD is
always valid JSON).
let you do it.
However if you know your mapping from XML to JSON-LD then you could
implement something at the same level as
org.exist.util.serializer.TEXTWriter to output your JSON-LD.
--
Adam Retter
eXist Developer
{ United Kingdom }
irc://irc.freenode.net/existdb
Adam Retter
2017-04-22 11:01:10 UTC
Permalink
I think we might be talking at cross-purposes or certainly there is
some confusion.
Post by Christopher Johnson
Sure, it is possible to write JSON-LD as text, but this is not a proper
serialization of JSON.
There is valiid JSON and then there is plain text,
and they are not the same.
In the end it is all just bytes. The XML Serializer is really
outputting "text" (compliant to the XML spec) from XML! If you wrote a
JSON-JD serializer in a similar for to the TEXTWriter, it would simply
take RDF/XML as input and output "text" (compliant to the JSON-JD
spec).

JSON is also a specification of a format of text ;-)
Post by Christopher Johnson
For example, reading JSON as
unparsed text into a nodeset using XSLT,
fn:unparsed-text does not deliver a nodeset, it delivers an xs:string.
https://www.w3.org/TR/xpath-functions-30/#func-unparsed-text
Post by Christopher Johnson
and then sending the output to
json:contents-to-json will not yield valid JSON (at least for me).
I see that json:contents-to-json does take a node(), I am not sure how
you are going from xs:string (from output from fn:unparsed-text) to a
node() though.

I am either missing something, or this shouldn't be possible.
Post by Christopher Johnson
but rather in
providing a valid JSON-LD serialization (and RESTXQ) output method for any
RDF/XML that could be stored in an eXist db collection. I still suspect
that this could be a relatively common use case that is supported by already
well-developed libraries.
Yes exactly. If you have XML as input (e.g. RDF/XML) then you can
create a serializer in Java by using XMLWriter/TEXTWriter etc as
examples, as they all take XML as input and then apply a mapping to
whatever output format (in your instance you would want a mapping to
JSON-LD).

I just realised that we already have a serialization writer for a
basic mapping of XML -> JSON, see
org.exist.util.serializer.json.JSONWriter.

So you just need to create a
org.exist.util.serializer.json.JSONLDWriter which maps your XML/RDF to
your JSON-LD.
--
Adam Retter

eXist Developer
{ United Kingdom }
***@exist-db.org
irc://irc.freenode.net/existdb
Christopher Johnson
2017-04-23 15:12:23 UTC
Permalink
​​
Right, "Text serialization" is a very broad term.. Perhaps it is a bit
more clear to refer here to output streams (from a provider service) as
defined by a *media type*.
From what I can tell, the existing "mapping" logic for media type "json" is
done in the XQuerySerializer util. This calls the
org.exist.util.serializer.json.JSONSerializer
based on the SerializationProperty.METHOD value from the RESTXQ
implementation.

With these three components, I see a mapping workflow that involves this:
Defining a new media type constant for JSON-LD in a convenience class.
This constant could be used by a RESTXQ implementation to build an
XQueryStreamProvider that can type check whether a requested sequence is
writeable based on media type (i.e. SerializationProperty.METHOD value).
In my use case, the sequence needs to be RDF/XML (e.g. the root node
contains the xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" namespace
). If true, then the provider streams the sequence to a pre-built RDF/XML
to JSON-LD serializer, like the one previously referenced.

I am not clear on how (or if) an XQueryStreamProvider can type check a
sequence based on "contains namespace". Any suggestions?

Cheers,
Christopher
I think we might be talking at cross-purposes or certainly there is
some confusion.
Post by Christopher Johnson
Sure, it is possible to write JSON-LD as text, but this is not a proper
serialization of JSON.
There is valiid JSON and then there is plain text,
and they are not the same.
In the end it is all just bytes. The XML Serializer is really
outputting "text" (compliant to the XML spec) from XML! If you wrote a
JSON-JD serializer in a similar for to the TEXTWriter, it would simply
take RDF/XML as input and output "text" (compliant to the JSON-JD
spec).
JSON is also a specification of a format of text ;-)
Post by Christopher Johnson
For example, reading JSON as
unparsed text into a nodeset using XSLT,
fn:unparsed-text does not deliver a nodeset, it delivers an xs:string.
https://www.w3.org/TR/xpath-functions-30/#func-unparsed-text
Post by Christopher Johnson
and then sending the output to
json:contents-to-json will not yield valid JSON (at least for me).
I see that json:contents-to-json does take a node(), I am not sure how
you are going from xs:string (from output from fn:unparsed-text) to a
node() though.
I am either missing something, or this shouldn't be possible.
Post by Christopher Johnson
but rather in
providing a valid JSON-LD serialization (and RESTXQ) output method for
any
Post by Christopher Johnson
RDF/XML that could be stored in an eXist db collection. I still suspect
that this could be a relatively common use case that is supported by
already
Post by Christopher Johnson
well-developed libraries.
Yes exactly. If you have XML as input (e.g. RDF/XML) then you can
create a serializer in Java by using XMLWriter/TEXTWriter etc as
examples, as they all take XML as input and then apply a mapping to
whatever output format (in your instance you would want a mapping to
JSON-LD).
I just realised that we already have a serialization writer for a
basic mapping of XML -> JSON, see
org.exist.util.serializer.json.JSONWriter.
So you just need to create a
org.exist.util.serializer.json.JSONLDWriter which maps your XML/RDF to
your JSON-LD.
--
Adam Retter
eXist Developer
{ United Kingdom }
irc://irc.freenode.net/existdb
Adam Retter
2017-04-26 14:56:14 UTC
Permalink
From what I can tell, the existing "mapping" logic for media type "json" is
done in the XQuerySerializer util. This calls the
org.exist.util.serializer.json.JSONSerializer based on the
SerializationProperty.METHOD value from the RESTXQ implementation.
The SerializationProperty.METHOD just tells eXist which serializer to use.

I had forgotten actually that RESTXQ directly uses XQuerySerializer
which is a wrapper around a bunch of stuff including XMLWriter (or
it's subclasses).
XQuerySerializer just gives us the ability to deserialize the results
of an XQuery that are not just XML Nodes, as the result of an XQuery
is always a Sequence of values.

So in the case of JSONSerializer, for mapping XML to JSON its
serializeNode function actually defers to JSONWriter.

The mapping (XML->JSON) is actually done in the writer itself, i.e. JSONWriter.

So in this instance you would add a function to XQuerySerializer
called serializeJSONLD which would use a new JSONLDSerializer class
(which you would create and should be similar to JSONSerializer). In
the JSONLDSerializer class, you would also have a serializeNode
function, which would defer to a new JSONLDWriter class (which you
would create). The mapping from XML -> JSON-LD would be encoded in the
JSONLDWriter.


Hope that helps?
--
Adam Retter

eXist Developer
{ United Kingdom }
***@exist-db.org
irc://irc.freenode.net/existdb
peter winstanley
2017-04-20 14:12:43 UTC
Permalink
might this be more useful if it was added to something like https://github.com/ljo/exist-sparql ?
W.S. Hager
2017-04-21 08:09:26 UTC
Permalink
You don't need sparqls for JSON-LD. That was intentional.

As for serialization, what's the problem exactly? You just create a map
from XML and render JSON. As for namespaces, JSON-LD is not intended for
that. You don't need namespaces is JSON.
Post by peter winstanley
might this be more useful if it was added to something like
https://github.com/ljo/exist-sparql ?
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Exist-open mailing list
https://lists.sourceforge.net/lists/listinfo/exist-open
--
W.S. Hager
Lagua Web Solutions
http://lagua.nl
Loading...