proposal: encoding/yaml: Add YAML support in the standard library

[TheCoreMan]

TL;DR

Seems like since JSON and XML are supported in the standard library, it's time to move YAML support into the standard library as well.

Why now?

Following a discussion in the #cup-o-go channel in the Gopher slack, we figured that:

• YAML has a de-facto standard, so it CAN be added - it's not ambiguous
• YAML is massively used specifically in the Go space. The best example is K8s, Helm, etc. that's totally dominated by YAML. From grank.io:

• Since JSON, XML, and CSV are in the standard library, YAML can be part of the standard library as well
• Seems like it would be useful for other major projects in the Go space, such as Fyne on embedded chipsets

History

Seems like this proposal was suggested 13 years ago. YAML went a long way since then.

What do y'all think? Would any more data or research help?

[seankhliao]

which version of yaml? there's 1.2, which removes foot guns, but isn't widely implemented
what features should it support? k8s has its own fork for making structured edits, it's that something we will need to support too?

[earthboundkid]

Counterproposal: Block gopkg.in/yaml in the Go module proxy to stop the virus from spreading.

YAML is a terrible format which is never appropriate to use. There is literally always a better choice. Yes, it's popular in the devops and Go world, but that's no reason to encourage it.

[earthboundkid]

Less inflammatory statement: YAML has a large and complex spec. Putting it into the Go standard library means choosing a subset of the spec to implement. If we can’t implement a full YAML parser (and we really cannot), it makes more sense to leave it to third parties to pick and choose which subsets of YAML they want.

[mvdan]

Also note https://go.dev/doc/faq#x_in_std.

[sam-orca]

@carlmjohnson : I am not sure YAML is that complicated in comparison to XML. So the argument of being something is large and complex, we should not do it, is not really strong. And nothing wrong with just implement a subset of the specs first as well. XML is not come all at one, isnt it?
For example :
#11496 (7 years XML issue)

[soypat]

I've burned myself quite badly using YAML. YAML looks like a really approachable configuration language until you use it long enough to find the rough edges. It seems to me it has really bad design decisions marketed as easy to use, readable, which it is.

What worries me about adding YAML is that then there'll be more justification for using it over other better ML languages like TOML, JSON5 which are not yet in the standard library (heck, maybe I'd even give in to using it). I think if we do add YAML we should also have another ML similar language with better design so YAMLers can use their YAML and those who want a ML language, period-- can evaluate between YAML and a well designed ML language.

relevant: https://noyaml.com/

[woody1872]

Massive yes from me. I've wished for it so many times, I even started to write a draft proposal myself but never finished it, 616 words last modified on 6 Feb 2022. With the - some would say unfortunate - prevalence of YAML, support is really needed in the standard lib I feel, so that external imports become unnecessary. External imports can be difficult to justify in heavily regulated industries, and on occasion I've opted for JSON instead where perhaps YAML would have better/more consistent with other tooling and tech.

[woody1872]

Counterproposal: Block gopkg.in/yaml in the Go module proxy to stop the virus from spreading.

YAML is a terrible format which is never appropriate to use. There is literally always a better choice. Yes, it's popular in the devops and Go world, but that's no reason to encourage it.

Certainly agree YAML is horrible to work with, and I'd personally choose JSON over YAML and put up with the lack of comments. However, I think a dislike of YAML is not a good reason not to support it via the std lib, it's just so unbelievably prevalent these days to ignore.

[mvdan]

@sam-orca the fact that XML isn't being looked after the way it should in the standard library is probably an argument against adding more complex packages to the standard library. I suspect that, if it had to be considered for addition today, it wouldn't meet the bar outlined in the FAQ.

[earthboundkid]

To echo what mvdan has said, encoding/xml has already caused security problems because it only implements a subset of XML, which makes it unsuited to use in SAML, for example. YAML would have much larger versions of the same problems. Look at https://ruudvanasseldonk.com/2023/01/11/the-yaml-document-from-hell. YAML has directives which make producing a proper parse essentially impossible. We would have to pick a subset, and that subset would end up being observably different from Python, gopkg.in/yaml, etc. There's no alternative where we just implement the full spec. There is no full spec. Just say no to YAML.

[doggedOwl]

Something being not perfect or not liked is not the measure that should be used.
Everytime something "teoritically perfect" is proposed, someone asks: "is there a real use case for this?" "how often does this come up"?
Well here we have the opposite of those proposals. something that is not perfect but it is widely used and nomater how much we may wish for, it's not going away.
XML, CVS, etc have their problems and their implementation can never be complete but I am surely grateful we have a useful standard implementation anyway. You say XML implementation is unsuited for SAML for example, well guess what its very suited for a lot of other things and it's being used (begrudedly) succesfully.
The same goes for YAML. While I avoid it as much as I can (same as xml) the reality is that it's a must have. Better to have a standard way than again maybe 10 different 3rd party implementations all with their pecularities.

[earthboundkid]

Better to have a standard way than again maybe 10 different 3rd party implementations all with their pecularities.

Why? I think it’s better to have several third party libraries carve out the subset of YAML they support and if you need a different subset, you pick a different library. What’s gained by standardizing something that resists standardization?

[rsc]

This discussion has been a bit heated. I just want to encourage everyone to try to take the temperature down a little. In particular, comments like #61023 (comment) are not constructive in any way, but that's not the only problematic comment here.

YAML is an important and useful data format. Among the generic data formats, it stands out for being far more human editable than JSON while still being appropriate for encoding data structures (unlike XML and CSV). In an ideal world, which perhaps would contain a somewhat different definition of YAML, having an encoding/yaml package in Go's standard library would make sense.

I have considered the question of a standard encoding/yaml package for well over a decade. On the one hand, it is widely used, including in Go's own infrastructure like x/website and many popular Go projects. It would certainly help the community to standardize on a single, robust package. On the other hand, while basic YAML looks trivial, in detail it is incredibly complex, far more so than JSON (which it embeds) or even XML (which attempts much less). A full YAML parser and printer is a very large amount of subtle code, one that the current Go team practically speaking cannot write or support. On top of that, @niemeyer has done an excellent job writing gopkg.in/yaml.v3, which as the top comment points out is the most widely used YAML package in the Go ecosystem. For that I am incredibly grateful, and I think we should continue to encourage use of that package.

I have occasionally considered defining a strict subset of YAML ("YAML: the good parts" or "YAML: what everyone thinks it means") that removes most of the unexplored dark corners and would be a better base for tooling like x/website and other projects. Almost no one needs the * notation for creating cyclic references, for example, and that causes "interesting" complexity bugs. But that too would be a lot of work, and it's unclear how much real adoption it would get, especially with a very good full YAML implementation available instead. And it wouldn't belong in the standard library, at least not to begin with.

For all these reasons, it seems clear to me that encoding/yaml is not feasible for the forseeable future and probably not even desirable either. Thanks again to @niemeyer for his excellent work on gopkg.in/yaml.v3 (and gopkg.in itself!). I encourage anyone who needs YAML support to use it, as the Go team does.

[flimzy]

IMO, the biggest advantage of having yaml (or any encoding, really) in the standard library is not to build the end-all be-all implementation, but to create the framework for standardization.

I think we can consider encoding/json as an example. There are many alternative implementations that have their own niches. But the majority of them still use the standard library's json.Marshaler and json.Unmarshaler interfaces, at least as options. This standardization is valuable, even when using third-party implementations. And most, if not all, use the standard library's json struct tag syntax. Implementation aside, these standarizations are valuable when interoperating between disparate libraries.

I think an argument could be made on the same grounds for including yaml in the Go standard library. However, it's likely not a particularly strong argument in this specific case, considering:

1. as already pointed out, gopkg.in/yaml.v3 is widely adopted, and thus largely, if not completely, serves this purpose already.
2. not all niche uses conform to the standard interfaces, even for JSON. They're even less likely to for YAML, given its nuances.

[rsc]

This proposal has been declined as infeasible.
— rsc for the proposal review group