 |
cello
JUCE ValueTrees for Humans
|
|
cello
JUCE ValueTrees for Humans
|
Classes for working with juce ValueTree objects.
Brett g Porter * brett.nosp@m.@bgp.nosp@m.orter.nosp@m..net
API docs available here
Cello is a set of C++ classes to work with ValueTree objects from the JUCE application framework.
This project has several overlapping goals:
A new Value type provides type safety (including transparent conversion from arbitrary C++ types and the JUCE var type used within ValueTrees), optional validator functions called on set/get, and implementation of all the in-place arithmetic operators (for numeric types).
Additional support classes to safely use ValueTrees across thread and process boundaries (including over TCP connections and named pipes) simplify those use cases.
The design of the classes also simplifies the implementation of applications where the internals can be both loosely and dynamically coupled together using a super fine-grained implementation of the Observer pattern to support a reactive programming style.
The Object type:
cello::Value or not, in a Pythonesque manner.cello interface.Cello is released under the terms of the MIT license.
cpr2323, used in products we worked on together at Artiphon.ValueTree SkepticI've been using the JUCE framework for over a decade now, but there's a major component of JUCE that never clicked for me as a developer — ValueTrees. This wasn't a problem for me until I changed jobs and started needing to work on a mature codebase that made significant use of them. This code makes efforts to hide some of the more cumbersome or repetitive aspects of integrating ValueTrees into an application, but that ValueTreeWrapper class still seemed like it required too much effort to work with; where I'm used to thinking in terms of objects that contain values, any time I needed to get near data that's stored in a ValueTree, it was impossible to avoid the awareness that I was always working through an API to perform operations on data that should just be directly manipulable, and while the wrapper class approach mitigated this to some extent, there was still more boilerplate code to write than seems good to me, as well as other places where the gaps around the abstraction were more obvious than I like.
I've always found that the only way for me to work through these kinds of issues when I encounter them is to sit down with a blank document in an editor and start enumerating the problems that I see with a system and use that as a guide to start thinking about ways that I can engineer around the parts that aren't my favorite, and sometimes how I can reframe my thinking to start seeing superpowers where I thought there were deficiencies.
One of my current teammates has expressed confusion that I wasn't immediately on board with ValueTrees, and his defense of them was key to my eventually starting this re-analysis. They give you:
...but at the cost (in comparison to using native POD or class/struct variables) of being:
var variant type.As I started listing the tradeoffs, I considered ways to work around the convenience and type-safety issues. I also reflected on the years in my career when I wrote far more Python code than I did C++, and many of these same charges can be filed against that language, which I love.
At one level, you can look at Python as being nothing but a bunch of associative arrays (or in python, dicts) with the ability to be manipulated dynamically by code. Once I started thinking in those terms, the project became much more interesting.
As frequently happens with me, these thoughts sat collecting dust in a document until I hit upon a name for the project — cello, short for 'cellophane' (since the code is wrapping a ValueTree)
In short, my goal was: create a set of C++ classes that I can derive my own classes from where member variables are stored transparently in JUCE ValueTrees instead of directly in those object instances, combining the comfort and simplicity of working with normal-looking C++ code with the benefits and tradeoffs of ValueTrees.
Something similar to:
juce::Identifier and a reference to a ValueTree that provides the actual storage; storing or retrieving the value through its variable needs to do so through the ValueTree API, but that's all kept out of sight.juce::var objects internally. cello::Value objects remove concerns about type-safety that vars introduce.++, --, +=, -=, *=, /=) defined.juce::VariantConverter struct has been defined.cello::Value objects only make sense as members of a class derived from cello::Object (below). The signature of the Value constuctor is:
...so at creation time, a value knows:
VariantConverter facility in JUCE, almost any type of data can be converted to/from the var variant type.So, declaring an instance of this type templated on int as a member of a cello::Object object would look like
We pass a reference to the owning object, the ID to use, and its default initial value. By convention, we use the same name for the member variable as for its Identifier in the ValueTree.
We define a macro in cello_value.h that's less cumbersome and less potentially error-prone to do the same thing:
...so the above declaration would be MAKE_VALUE_MEMBER (int, x, {});. Once a cello::Object containing this declaration is instantiated, you can manipulate that value almost exactly the same as if it were an actual instance of the underlying type ("almost exactly" here covers edge cases like sizeof giving different results, and probably others that I haven't considered yet):
By defining a template specialization of the juce::VariantConverter struct, you can store more complex value types by cleverly packing them inside one of the more interesting var variants that exist — in this example from the cello unit tests, we use the fact that an Array of vars is a var:
Then we define a class that has a single public Value member that contains a std::complex<float> — there's no additional work required to perform the conversions:
Your code is then free to work with that value directly:
If we're taking some inspiration from Python here, it's worth remembering that Python developers are in the practice of leaving all their class member variables public instead of hiding them behind a wall of privacy and forcing the usage of getVariable()/setVariable() methods to ensure the separation of interface from implementation—much of the time, there's no reason to require those accessor/mutator methods, and when there is an actual reason (for example, to ensure the maintenance of a class invariant), it's easy to switch over to using a property to manage access to the underlying data. Bertrand Meyer, creator of the Eiffel programming language refers to this as the "Uniform Access Principle," that "...all services offered by a module should be available through a uniform notation, which does not betray whether they are implemented through storage or through computation."
Each cello::Value object may have ValidatePropertyFn lambdas assigned to it (where that lambda accepts a const reference to T and returns a T by value) that are (onSet) called before that value is stored into the underlying ValueTree or (onGet) called after retrieving the property from the ValueTree but before returning the value to calling code.
Your application can use this facility to modify the value (e.g. to keep it within a valid range), create an entirely new value, make changes to other properties of the ValueTree, create log entries, or anything else that you need to happen at these juncture points.
There will be times when a value stored in a ValueTree/Object needs to be used frequently enough that the overhead of re-fetching from the underlying tree and performing validation on it become problematic. The cello::Value::<T>::Cached class provides a simple mechanism to maintain a copy of a Value object that's automatically updated each time it changes.
The normal behavior of ValueTrees is to only notify callback listeners of property changes when a value actually changes. In practice, it's frequently useful to ensure that any attempt to set a property results in notifications being sent even if setting it to its current value. This can be controlled on a per-value basis by calling that value's forceUpdate (bool shouldForceUpdate) method.
To simplify the common case where this behavior is only meant to be in force for a single update, we provide a utility class ScopedForceUpdater that sets the value to force sending updates when the class is constructed, and then clears the update logic when that updater object goes out of scope.
It's also common to want to send update callbacks to all listeners except one—for example, if I have a bit of code that's setting a value and that code is also listening to the value, there's no need to receive a callback; that code already knows what the new value is. The cello::Value class provides a method void excludeListener (juce::ValueTree::Listener* listener) for this purpose.
Since our objects rely on separate ValueTree objects for their storage, we need to support two different mechanisms for creating instances:
The constructors of cello::Object handle both these cases for us, using the logic outlined below:
Object (const juce::String& type, Object* state); (preferred)Object (const juce::String& type, Object& state); (preferred)Object (const juce::String& type, juce::ValueTree tree);state or tree argument is of type type, wrap that inside the object being created.state or tree arguments has a child of type type, wrap that child inside the object beng created.type and initialize it as appropriate. If the state arg was not null (or the tree is valid), add this new tree as a child.It is sometimes useful to know whether a new Object was created or wrapped — for example, it might be an error in your application if a child that's expected to be present isn't.
You can test this at runtime using the method Object::getCreationType(), which will return either:
Object::CreationType::initializedObject::CreationType::wrappedThe type argument to an Object constructor can be richer than just a simple juce::Identifier; it's useful in an application to be able to pass around a single top-level context object and have individual objects within that hierarchy be able to find themselves.
Consider a common pattern where an application needs to have separate trees to collect persistent attributes that are saved and restored between app runs, and another that holds runtime values that are recreated each time the application runs. Rather than write procedural code to start at the root and find (or create) individual child objects that are expected, we can do the same thing declaratively using paths, like: /persistent/object1/object2, which would start at the root tree, then descend through children persistent, object1, and object2, creating any children that are not found.
Path elements are separated with forward slashes.
If the first character in a path string is /, the path is absolute starting at the root.
All other paths are relative to the Object that's passed into an Object constructor.
Path elements starting with a circumflex character ^ will search upward from the current path location to find an ancestor Object of a specified type, so ^grandpa is read as "search upward in the hierarchy from the current path location until you find an object of type `grandpa`".
A path element of .. operates as it does in file systems, navigating to the parent of the current path location, so ../sibling would find a sibling object of the current one, and ../../uncle will look for a sibling of the current object's parent.
All other paths must be valid juce::Identifiers, and search downward through child objects.
Downward searches when instantiating cello::Objects will create child trees (that will not be initialized) as needed.
Searches upward from an object will not be able to create interim object/trees. You can check the CreationType after the constructor executes to make sure that you have a valid object before using it. You can test for existence before attempting creation by instantiating a cello::Path object directly and using its findValueTree() method with a search type of Query; if that search returns an invalid juce::ValueTree, you'll need to handle that case as appropriate, whether it's an error, or just triggers additional configuration/creation of the hierarchy before using it.
ValueTrees can contain other ValueTrees as children, and it's important to keep in mind that there are two different modes for this containment:
There's no mechanism to enforce this distinction—if a list of different types makes sense in your application, there's a little more logic you'll need to write, but that's all.
void append (Object* object); adds the child to the end of this object's child list.
void insert (Object* object, int index); adds the child at a specific index in the list; if index is out of range (less than zero or greater than the current number of children), the child will be appended to the list.
To remove a child that's already wrapped in an Object, use
On success, this will return the same pointer you passed it; if that Object was not actually a child of this object, will return nullptr.
To remove a child by its index:
This will return the raw ValueTree used by that child on success, or an invalid ValueTree on failure.
We provide an operator[] to access children by their index:
You can also iterate through an Object's children:
You can change the position of an individual child using the method
...and sort all the children with the method:
where Comparator is an object that contains a method
that returns
The stableSort argument specifies whether the sort algorithm should guarantee that equivalent children remain in their original order after the sort.
After cello release 1.1, you may wish to instead use the new database/query features for searching and sorting.
Use the cello::Query object to define a set of search and sort criteria to use to perform simple database-like operations. Instead of defining a query language, we've defined two function types that can be passed into a Query object to define its behavior at run time:
You can specify any number of predicate functions for a Query object to use; these functions accept a ValueTree as an argument and return a boolean to indicate whether this child tree should be included in the query search results.
The search logic will execute these functions in the order they were added until encountering one that returns false. If all of the query predicates return true, a copy of this ValueTree will be added to the search results.
If a query is run with no predicate functions defined, all children of the Object being searched will be copied and added to the search results.
You can also specify comparison functions that will be used to sort the results list after a query is performed; if none are provided, the items in the search results will be in the same order they exist in the Object being queried.
These use a concept borrowed from the MongoDB NoSql database; an 'upsert` operation performs one of:
The main use case here would be to
For this to work, your items must be defined such that each has a unique key value that can be used to link the update tree with the original one to be updated. In the unit tests for this function, our Data objects have an attribute key that is populated with a monotonically incremented integer when created. In production code, it would be better to use something more unique, like a juce::Uuid.
Most ValueTree operations accept a pointer to a juce::UndoManager object as an argument to make those operations undoable/redoable. cello::Objects can maintain this manager for you: pass a pointer to UndoManager to a cello::Object using its setUndoManager method, and that object and any child/descendant objects that are added to it will become undoable.
The following undo/redo methods are available directly from cello::Object:
bool canUndo () const;bool undo ();bool canRedo () const;bool redo ();void clearUndoHistory ();You can also retrieve a pointer to the UndoManager (using juce::UndoManager* getUndoManager()) for any of its other operations that we don't expose directly.
The cello::Object class defines a set of std::functions that can be installed as callbacks to be executed when properties or children of an object are changed:
PropertyUpdateFn signature: std::function<void(juce::Identifier)>
You can register a callback for each named property of an object that will be executed when the value of that property is changed. You can also register a wildcard callback using the identifier of the object itself that will be called when an attribute changes but there was no specific handler for it.
There are two Object methods to register these callbacks:
void onPropertyChange (juce::Identifier id, PropertyUpdateFn callback) — pass in the identifier of the attribute to watchvoid onPropertyChange (const ValueBase& val, PropertyUpdateFn callback); — pass in a reference to the cello::Value or cello::Object to watch.If the Value that you're watching is a public member of an Object, you can also subscribe to its updates directly using the method Value<T>::onPropertyUpdate (PropertyUpdateFn callback);
Changes to children are broadcast using a ChildUpdateFn callback that has the signature std::function<void (juce::ValueTree& child, int oldIndex, int newIndex)>;
onChildAdded — oldIndex will be -1, newIndex will be the index of the new child.onChildRemoved — oldIndex will be the index of the child that was removed, newIndex will be -1.onChildMoved — oldIndex and newIndex are self-explanatory.A SelfUpdateFn callback with the signature std::function<void (void)> will be called when:
onParentChanged — this object has been adopted by a different parent tree.onTreeRedirected — the underlying value tree used by this object was replaced with a different one.Not everything can or should be done with the kind of compile-time API cello was written to support. These methods take their names and inspriation from similar methods in the Python object model.
These methods do provide some level of type-safety and type-coercion using VariantConverters that our Value types have.
bool hasattr (const juce::Identifier& attr) const tests an object to see if it has an attribute/property of the specified type (enabling what the Python world would call 'Look Before You Leap' programming)template <typename T> Object& setattr (const juce::Identifier& attr, const T& attrVal); sets the value of the specified attribute in the object. We return a reference to the current Object so that multiple calls to this method can be chained together.template <typename T> T getattr (const juce::Identifier& attr, const T& defaultVal) const either returns the current value of the specified attribute, or a default value if it's not present.cello::Object instances can be persisted to or from disk in any of the three formats that ValueTrees support:
To save a file, use the bool save (juce::File file, FileFormat format = FileFormat::xml) const method, which will write out that tree and all its descendants into the specified file.
Loading a file is a little more complex; we use a static method static juce::ValueTree load (juce::File file, FileFormat format = FileFormat::xml) that attempts to load and return a ValueTree from the specified file; you should then pass that ValueTree (if valid) to the constructor of your application's root Object type and verify that the constructor was able to wrap the tree it was given, code like:
When working with multiple threads, it's important to ensure that when two threads work with the same piece of data that they do so using techniques that prevent the common problems when using threads—race conditions, data corruption, deadlocks, etc.
cello provides the cello::Sync class to support clean updates across thread boundaries. We do this using a pair of cello::Objects of the same underlying ValueTree type, letting the juce::ValueTreeSynchroniser object perform most of the hard work: when the 'producer' object is changed, it generates a small binary payload containing the deltas that need to be applied to the consumer object to make them sync up. Because all the operations are performed on the ValueTrees themselves, once a Sync object is created to connect the pair, your code doesn't need to concern itself over the origin of a change.
cello::Sync objects are created with this constructor:
To perform bidirectional sync operations, create a pair of Sync objects with the products/consumer roles swapped appropriately. You'll need to be careful when doing this to avoid creating feedback loops where updates echo infinitely between Objects.
When the consumer object is being updated on the message thread, the Sync class will handle executing the updates automatically for you. Consumers being updated in a worker thread will need to find a place in their run() loop to check for and execute any pending updates. A minimal worker thread class would look something like:
There are parts of the juce::ValueTree API that are not available through the cello API; these may be added later, or you can use them directly by accessing the ValueTree object that an Object already owns.
There is a separate repo containing a small unit test runner; you can also add my testSuite JUCE module as a component in your application to execute the tests in your own app.
See CHANGELOG
1.13.2