JSON Schema for farm management data: question about vocabularies / definitions

[gbathree]

Hello - first time poster. I work at our-sci.net (part of openteam.community) and we're working to create a flexible and transparent data structure for farm management data that encourages data transparency and reusability. We're all about open source and public, appropriately permissioned data.

Using JSON Schema, we've built a gitlab CI pipeline to enable easy creation, modification, and forking (hopefully with inheritance) of json schema's. In essence, we want this to be a an accessible publication pipeline to answer the question 'hey... how do you store / understand farm management data?'. Right now, the publication outputs are a set of schemas in fixed URLs using gitlab artifacts, human readable documentation using gitlab pages (we'll be switching to docusaurus in the future probably), and published NPM packages which validate your published schemas. In ensuring this publication process is public and transparent and easily fetchable, (we hope!) it will actually reduce re-inventing farm management data structure designs by making it really easy to see and use smart designs by others.

It's still in alpha (under active development and not complete), but here's the link: https://gitlab.com/our-sci/software/json_schema_distribution.

None of us are JSON Schema experts, but we're learning fast. As we go, we are trying hard to use the best features and norms developed by the JSON Schema core community (which - thank you so much - there really aren't any other tools like these out there, so massive kudos for all the work!). However, sometimes we get confused and/or stuck, and I'm always worried we're somehow not quite doing it right or in the best way possible.

I thought about diving into my questions here (which I could do) but I also wondered if there were places where groups could present their projects broadly and get feedback about their use of JSON Schema. I think that would be massively valuable for us. We (at OpenTEAM) also have In Depth Webinars where community members or community-adjacent groups present, and it may be valuable for the broader community to just learn more about JSON Schema and how it can be used (maybe some good practical examples).

I think a presentation to OpenTEAM would help a lot actually - it's been my experience that when I talk to developers about using JSON Schema, they tend to find it overly complex and ask 'Why can't I just do this in ... who cares if it's done in this schema way'... and I think I sometimes struggle to explain the long-term value that's so important to me - transparency, comparability, ease of cross-platform / language use, data migration, etc. etc.

Anyway, sorry a bit rambling, but in short - is it possible to 1) present our project and get feedback and 2) have a core person from your json-schema.org present to OpenTEAM?

[Relequestual]

Hey @gbathree, thanks for reaching out.

I gave a talk last year which covered four case studies: https://www.youtube.com/watch?v=48QgjmwLWh0
You can find a series of six video based case studies: https://www.youtube.com/watch?v=fkziMQD7pqQ&list=PLHVhS4Tj1YZOrrvl7_a9LaBAtst7BWH8a
You can find further written case studies on our blog: https://json-schema.org/blog

You're more than welcome to come join our slack server: https://json-schema.org/slack

is it possible to present our project and get feedback

Sure! We don't have any specific space for this. We have "office hours", but it's not well attended. IMHO, you'd be best off just sharing here or in the slack server.

is it possible to have a core person from your json-schema.org present to OpenTEAM?

I guess it could be possible, although it's not something we have done previously. We may be less well funded and organized than you imagine. What would you want to be presented? What would you want to learn? =]

[gbathree]

Hey Ben, thanks for your detailed response. Let me review the material you shared, and then I'll respond to your questions about presenting. Thanks so much for your response!

[gbathree]

Very helpful Ben - I passed this along to the FarmOS community. I have another question.

We're having a general conversation comparing JSON Schema, JSON-LD and RDF. (If you check this soon before more conversation fills in afterwards, it's here between Mike and Paul mostly --> https://app.element.io/#/room/#farmOS:matrix.org).

I think the value of adding local or global level semantic relationships using JSON-LD to data with an associated JSON Schema could be really helpful... however, I also see that there are ways to do similar things using JSON Schema with #defs or I've seen in vocabularies.

Maybe this is just a case of there's many ways to do the same thing, and there's no compelling reason to choose one over another... or maybe there are many real meaningful differences. Do you have any opinions (or pointers to discussions) around this topic?

This may be silly, but I had a fantastic conversation with ChatGPT about this, curious if this tracks with your thinking --> https://chat.openai.com/share/13d15bc9-c30e-4fb9-a960-6ecdf2e5ddbb

[jdesrosiers]

Your ChatGPT conversation was very typical ChatGPT in that some of it was great, some of it was ok, and some of it was completely off the rails. So, let me try to clarify a few things.

JSON-LD and RDF are not distinct choices. JSON-LD is RDF represented in a JSON compatible format. So, any place where it makes an argument for using one over another is nonsense.

It mostly did a good job at representing that JSON Schema and JSON-LD are distinct things that do different things with little to no overlap. JSON Schema describes the structure of a data structure such as what properties it includes. JSON-LD defines what those properties mean. You can't use JSON-LD to define the structure of a data structure and you can't use JSON Schema to define the meaning of property. It was way off the rails showing you a JSON Schema as a JSON-LD document. What you're looking for is something more like this.

You don't seem to quite have the right idea about what $defs and vocabularies are about. It sounds like you're talking about providing reusable schemas. $defs are generally used for schema reuse within the schema they are defined in. However, all schemas have a URI identifier. If you want a schema to be sharable, generally you would define it as it's own schema and it can be used by referencing its URI. Vocabularies are for defining custom JSON Schema keywords. For example, let's say you want to ensure that a date value is greater the given date. There's no way to express that constraint in JSON Schema, so you'd have to invent a new keyword to do it. You would use vocabularies to define that new keyword. Vocabularies doesn't solve the kind of think you're asking about.

In the end, I agree with ChatGPT's advice to start with JSON Schema and then define an ontology for your domain that can be used with JSON-LD if you determine there's a need for it.