yaml_resources.md

layout	page
title	YAML Resources
group	listed_pages

YAML, what is it good for?

YAML (YAML Ain't Markup Language) has been a structured data format frequently used in of Ruby development for some time. It has many useful features that often go unnoticed. Its ability to store serialized objects has recently made much news in the form of very serious vulnerabilities in the Rails framework.

Let's explore ways of using YAML beyond a simple store of arrays and hashes of data, as well as the risks and benefits of doing so.

We will discuss YAML databases, configuration files, conversion into and from Ruby objects, incompatibilities between parsers (Syck and Psych) and the reasons for them, and some gotchas.

Some reference will be made to YAML libraries in Ruby and other languages. http://www.yaml.org/

In progress notes on the topic follow

YAML Spec and Terminology

• YAML Ain’t Markup Language (YAML™) Version 1.2

{% highlight yaml %} %YAML 1.2

YAML: YAML Ain't Markup Language

What It Is: YAML is a human friendly data serialization standard for all programming languages. {% endhighlight %}

• Symbols
• Classes
• JSON superset

The primary objective of this revision is to bring YAML into compliance with JSON as an official subset. YAML 1.2 is compatible with 1.1 for most practical applications - this is a minor revision. An expected source of incompatibility with prior versions of YAML, especially the syck implementation, is the change in implicit typing rules. We have removed unique implicit typing rules and have updated these rules to align them with JSON's productions. In this version of YAML, boolean values may be serialized as “true” or “false”; the empty scalar as “null”. Unquoted numeric values are a superset of JSON's numeric production. Other changes in the specification were the removal of the Unicode line breaks and production bug fixes. We also define 3 built-in implicit typing rule sets: untyped, strict JSON, and a more flexible YAML rule set that extends JSON typing.

Terms

Collections
  Sequence
  Mapping

  Mapping-in-Sequence Shortcut
  Sequence-in-Mapping Shortcut

  Merge key

Basic Types
  Strings
  Indicators in Strings
  Plain scalars
  Null
  Boolean
  Integers
  Integers as Map Keys
  Floats
  Time
  Date
  Blocks
  Single ending newline
  The '+' indicator
  Three trailing newlines in literals
  Extra trailing newlines with spaces
  Folded Block in a Sequence

Aliases and Anchors

Documents
Trailing Document Separator
Leading Document Separator
YAML Header

YAML For Ruby
Symbols
Ranges
Regexps
Perl Regexps
Struct class
Nested Structs
Objects
Extending Kernel::Array
Extending Kernel::Hash

YAML Ruby Library Code

• YAML MRI 2.0.0
• YAML MRI 1.9.3
• YAML MRI 1.8.7

Psych and Syck

• Psych isn't Syck
• Psych vs. Syck

Guides

• YAML homepage
• Complete idiot's guide to YAML (config.yml)
• YAML Tutorial 2
• YAML Cookbook

Gotchas

• Yaml, Psych and Ruby 1.9.2-p180 – Here there be dragons

Ruby

• 1.8 : uses Syck
• 1.9 YAML == Psych, but can YAML::ENGINE.yamler = 'syck'
• 2.0 YAML == Psych, Syck removed from stlibi

Tricks

{% highlight yaml %} defaults: &defaults adapter: mysql2 encoding: utf8 reconnect: false pool: 5 username: sqluser password: s3cret host: localhost

development: <<: *defaults database: app_development

test: &test <<: *defaults database: app_test

production: <<: *defaults username: productionsqluser password: productions3cret database: app_production {% endhighlight %}

Security

• What this means for your startup

• YAML F7U12

• Rails' Remote Code Execution Vulnerability Explained

• Analysis of Rails XML Parameter Parsing Vulnerability

• Reddit Discussion Of: Serious vulnerability in Ruby on Rails allowing arbitrary Ruby code execution in any Rails application (groups.google.com)

• HackerNews discussion of POC

• Psych issue to consider adding #safe_load

• [SEC][ANN] Rails 3.2.12, 3.1.11, and 2.3.17 have been released!

• Multiple vulnerabilities in parameter parsing in Action Pack (CVE-2013-0156)

• Rails PoC exploits for CVE-2013-0156 and CVE-2013-0155

• Metasploit Framework: Ruby on Rails XML Processor YAML Deserialization Scanner

• Serialization Mischief in Ruby Land (CVE-2013-0156)

• Exploiting Ruby on Rails with Metasploit (CVE-2013-0156)

• Analysis of Rails XML Parameter Parsing Vulnerability and the commit that introduced it

Interesting

• YAML Support and Hash Representer In representable-1.2.7!
• Putting YAML at the top of a Markdown file with Metadown

Misc dump for now

• YAML is terrible and should be driven from the face of the Earth.
• Psych 2.0
  • Psych.safe_lod commit
  • Whitelistied types
• Rails 4 uses Josh Peek's sprockets-rails 2 manifest files are now json, not yaml. In communication: JSON has better support in non-Ruby languages.
• How RubyGems.org patched Psych/Syck following the 2013 January attacks
• Rails Allow to_xml and from_xml to work with AR models that have serialized fields
• Psych: Strings that look like dates should be treated as strings and not dates.
• Fun with Syck and Psych on Heroku. This is one of those posts that's inspired by a desire never to have to investigate the same bug again.
• Psych-based yaml in Ruby 1.9.3 too slow
• Psych can't parse YML in RefineryCMS that syck can
• Delayed job 3.0.1 not working, breaks on to_yaml
• Convert Syck to Psych YAML format
• [Bug #3112] require "yaml" doesn't use psych as default
• [ruby-trunk - Feature #6163][Open] Remove syck YAML extension
• Removing Syck from ruby
• VCR: Request/Response data is saved to disk as YAML by default
  • YAML issues in VCR
  • hiding vcr recorded api credentials
• Parsing YAML 1.1 with Ruby
• Shaving A YAML Yak
• Force YAML to use syck rather than psych. Eventually delayed_job should be made compatible with both.
• work around the syck/psych confusion yaml brings to ruby 1.9.2 by trying to load psych before yaml
• SafeYAML: Support for custom deserializer?
• JRuby 1.9 problem with Syck
• Q&A
  • Why the difference in behavior of YAML parsers (syck and psych)?
  • rails error, couldn't parse YAML