Post-SciPy 2024 tutorial notes

[jonmmease]

@ChristopherDavisUCI and I gave a Vega-Altair tutorial at SciPy 2024 Monday morning. The tutorial materials are at https://github.com/vega/SciPy2024-Altair-Tutorial. I just wanted to record some notes on potential Vega-Altair features / fixes that participants brought up.

1. expressions are hard to debug: On participant ran into a situation when using the filter transform where they misspelled one of the columns in a datum.{col} expressions. This doesn't result in an error, as Vega (and VegaFusion) follow JavaScript's semantics where this expressions returns undefined rather than erroring. This resulted in the entire filter expression coercing to false, and all of the data being dropped. Making this an error in Vega would be a breaking change, but I think we could make VegaFusion more strict about this.
2. better type safety in aggregation expressions: On participant asked if it would be possible for the type checker to catch errors in the aggregation string in an expression like alt.Y("countt()"). I didn't know off hand whether there's a more general form possible here that would type check. Or perhaps we could write a mypy plugin that would validate these aggregation strings.
3. desire to scale size of interacive charts: One participant asked if it were possible to scale up the physical size of charts. This is possible using the png renderer by setting scale_factor to 2+, but I don't think it's possible to accomplish this with an interactive chart. This would need to be implemented in Vega if it's not already possible.
4. support PIL images: One participant wanted to make a chart with an image mark like this, but their images were stored locally as PIL Images. I explained that it's possible to convert PIL images to base64 encoded PNG urls, but this is a bit of an annoying process. In plotly, I added support for doing this automatically during serialization and I think we could do the same thing in Vega-Altiar. Here's the pil_image_to_uri utility function I added in plotly for this conversion.
5. self-hosted Vega editor: Someone asked if it's possible to host the Vega editor yourself. And if so, whether it's possible to point Altair's to_url() and open_editor() methods to this instance. I'm thinking it could make a lot of sense to have Altair check for a VEGA_EDITOR_URL environment variable, and it found use this for these methods. I'm also not sure if it's currently possible to override the editor URL in the Open in Editor dropdown entry.

I can turn these into issues later, but just wanted to document the notes quickly while they were fresh in my mind.

[joelostblom]

Thanks for sharing the material and some of the takeaways from Scipy!

Making this an error in Vega would be a breaking change

Although it is breaking, it sounds quite useful to receive an error instead of the expression silently not doing what's expected, so maybe it could be considered for a future major Vega release?

[dangotbanned]

2. better type safety in aggregation expressions:
   On participant asked if it would be possible for the type checker to catch errors in the aggregation string in an expression like alt.Y("countt()").
   I didn't know off hand whether there's a more general form possible here that would type check.
   Or perhaps we could write a mypy plugin that would validate these aggregation strings.

@jonmmease I was holding off on this until an issue was opened, but I've rolled it into a longer-term RFC as the agg class in #3505.

I thought it might be helpful sharing here how I arrived there, as it is an alternative solution for the same problem.

---

Why not a mypy plugin?

There are several other type checkers spec, discussion and this would only work for mypy users.
I'm also unsure how an optional dependency would work in annotations. Especially since they would no longer be str.

If not mypy then what?

I think typing.Annotated(PEP 593) would be something to explore.

Some examples from popular libraries:

• msgspec
• pydantic

So if we say Annotated works for this, the next issue is what does such an annotation actually look like?

Existing code

Building up the patterns

altair/altair/utils/core.py

Lines 196 to 204 in 679a7ce

	SHORTHAND_UNITS = {
	"field": "(?P<field>.*)",
	"type": "(?P<type>{})".format("|".join(VALID_TYPECODES)),
	"agg_count": "(?P<aggregate>count)",
	"op_count": "(?P<op>count)",
	"aggregate": "(?P<aggregate>{})".format("|".join(AGGREGATES)),
	"window_op": "(?P<op>{})".format("|".join(AGGREGATES + WINDOW_AGGREGATES)),
	"timeUnit": "(?P<timeUnit>{})".format("|".join(TIMEUNITS)),
	}

Inference, additionally factoring in the `Chart.data`

altair/altair/utils/core.py

Lines 503 to 687 in 679a7ce

	def parse_shorthand(
	shorthand: dict[str, Any] | str,
	data: pd.DataFrame | DataFrameLike | None = None,
	parse_aggregates: bool = True,
	parse_window_ops: bool = False,
	parse_timeunits: bool = True,
	parse_types: bool = True,
	) -> dict[str, Any]:
	"""
	General tool to parse shorthand values.
	These are of the form:
	- "col_name"
	- "col_name:O"
	- "average(col_name)"
	- "average(col_name):O"
	Optionally, a dataframe may be supplied, from which the type
	will be inferred if not specified in the shorthand.
	Parameters
	----------
	shorthand : dict or string
	The shorthand representation to be parsed
	data : DataFrame, optional
	If specified and of type DataFrame, then use these values to infer the
	column type if not provided by the shorthand.
	parse_aggregates : boolean
	If True (default), then parse aggregate functions within the shorthand.
	parse_window_ops : boolean
	If True then parse window operations within the shorthand (default:False)
	parse_timeunits : boolean
	If True (default), then parse timeUnits from within the shorthand
	parse_types : boolean
	If True (default), then parse typecodes within the shorthand
	Returns
	-------
	attrs : dict
	a dictionary of attributes extracted from the shorthand
	Examples
	--------
	>>> import pandas as pd
	>>> data = pd.DataFrame({"foo": ["A", "B", "A", "B"], "bar": [1, 2, 3, 4]})
	>>> parse_shorthand("name") == {"field": "name"}
	True
	>>> parse_shorthand("name:Q") == {"field": "name", "type": "quantitative"}
	True
	>>> parse_shorthand("average(col)") == {"aggregate": "average", "field": "col"}
	True
	>>> parse_shorthand("foo:O") == {"field": "foo", "type": "ordinal"}
	True
	>>> parse_shorthand("min(foo):Q") == {
	... "aggregate": "min",
	... "field": "foo",
	... "type": "quantitative",
	... }
	True
	>>> parse_shorthand("month(col)") == {
	... "field": "col",
	... "timeUnit": "month",
	... "type": "temporal",
	... }
	True
	>>> parse_shorthand("year(col):O") == {
	... "field": "col",
	... "timeUnit": "year",
	... "type": "ordinal",
	... }
	True
	>>> parse_shorthand("foo", data) == {"field": "foo", "type": "nominal"}
	True
	>>> parse_shorthand("bar", data) == {"field": "bar", "type": "quantitative"}
	True
	>>> parse_shorthand("bar:O", data) == {"field": "bar", "type": "ordinal"}
	True
	>>> parse_shorthand("sum(bar)", data) == {
	... "aggregate": "sum",
	... "field": "bar",
	... "type": "quantitative",
	... }
	True
	>>> parse_shorthand("count()", data) == {
	... "aggregate": "count",
	... "type": "quantitative",
	... }
	True
	"""
	from altair.utils.data import is_data_type
	if not shorthand:
	return {}
	patterns = []
	if parse_aggregates:
	patterns.extend([r"{agg_count}\(\)"])
	patterns.extend([r"{aggregate}\({field}\)"])
	if parse_window_ops:
	patterns.extend([r"{op_count}\(\)"])
	patterns.extend([r"{window_op}\({field}\)"])
	if parse_timeunits:
	patterns.extend([r"{timeUnit}\({field}\)"])
	patterns.extend([r"{field}"])
	if parse_types:
	patterns = list(itertools.chain(*((p + ":{type}", p) for p in patterns)))
	regexps = (
	re.compile(r"\A" + p.format(**SHORTHAND_UNITS) + r"\Z", re.DOTALL)
	for p in patterns
	)
	# find matches depending on valid fields passed
	if isinstance(shorthand, dict):
	attrs = shorthand
	else:
	attrs = next(
	exp.match(shorthand).groupdict() # type: ignore[union-attr]
	for exp in regexps
	if exp.match(shorthand) is not None
	)
	# Handle short form of the type expression
	if "type" in attrs:
	attrs["type"] = INV_TYPECODE_MAP.get(attrs["type"], attrs["type"])
	# counts are quantitative by default
	if attrs == {"aggregate": "count"}:
	attrs["type"] = "quantitative"
	# times are temporal by default
	if "timeUnit" in attrs and "type" not in attrs:
	attrs["type"] = "temporal"
	# if data is specified and type is not, infer type from data
	if "type" not in attrs and is_data_type(data):
	unescaped_field = attrs["field"].replace("\\", "")
	data_nw = nw.from_native(data, eager_or_interchange_only=True)
	schema = data_nw.schema
	if unescaped_field in schema:
	column = data_nw[unescaped_field]
	if schema[unescaped_field] in {
	nw.Object,
	nw.Unknown,
	} and is_pandas_dataframe(nw.to_native(data_nw)):
	attrs["type"] = infer_vegalite_type_for_pandas(nw.to_native(column))
	else:
	attrs["type"] = infer_vegalite_type_for_narwhals(column)
	if isinstance(attrs["type"], tuple):
	attrs["sort"] = attrs["type"][1]
	attrs["type"] = attrs["type"][0]
	# If an unescaped colon is still present, it's often due to an incorrect data type specification
	# but could also be due to using a column name with ":" in it.
	if (
	"field" in attrs
	and ":" in attrs["field"]
	and attrs["field"][attrs["field"].rfind(":") - 1] != "\\"
	):
	raise ValueError(
	'"{}" '.format(attrs["field"].split(":")[-1])
	+ "is not one of the valid encoding data types: {}.".format(
	", ".join(TYPECODE_MAP.values())
	)
	+ "\nFor more details, see https://altair-viz.github.io/user_guide/encodings/index.html#encoding-data-types. "
	+ "If you are trying to use a column name that contains a colon, "
	+ 'prefix it with a backslash; for example "column\\:name" instead of "column:name".'
	)
	return attrs

Thoughts

If parse_shorthand used a simple regex pattern, I would be more in favour of exploring this.

Not saying it couldn't be solved, but what we have is pretty complex and somewhat obfuscated.

Alternative solution

• feat(RFC): Adds agg, field utility classes #3505

I approached this quite differently, but it aligns well with upcoming changes to expr from #3466 and proposed changes in #3239

Examples

Aggregation functions are classmethods accessible under agg.
This simplifies the problem as incorrect method names are a runtime error:

These all have the same 2 arguments, both of which are optional.
Each method docstring could also provide examples on usage, something that wouldn't be feasible before

Auto-complete on the type argument

Both static AND runtime checking, so even non-typing users get some benefits:

A range of examples:

Backwards-compatible string syntax via alt.agg() like alt.expr():

Integrating this was an incredibly minor change in 5929a5e (#3505)

[jonmmease]

Thanks for sharing @dangotbanned, this looks like a very nice approach, and consistent with the expression language. I'm in favor!