Let’s train the network via gradient descent. JAX’s random number generator is set up differently than Numpy’s, so to initialize network parameters we’ll use the original Numpy library (onp) to generate random numbers. We’ll also import the tree_multimap utility to easily manipulate collections of per-parameter gradients (for TensorFlow users, this is analogous to nest.map_structure for Tensors). @@ -134,7 +134,7 @@
Evaluating our network again, we see that the sinusoid curve has been correctly approximated.
-
This result is nothing to write home about, but in just a moment we’ll re-use a lot of these functions to implement MAML.
@@ -158,7 +158,7 @@Now let’s extend our sinusoid regression task to a multi-task problem, in which the sinusoid function can have varying phases and amplitudes. This task was proposed in the MAML paper as a way to illustrate how MAML works on a toy problem. Below are some points sampled from two different tasks, divided into “train” (used to compute the inner loss) and “validation” splits (sampled from the same task, used to compute the outer loss).
-
$\mathcal{L}(\theta - \nabla \mathcal{L}(\theta, x_1, y_1), x_2, y_2)$ MAML’s inner update operator is just gradient descent on the regression loss. The outer loss, maml_loss, is simply the original loss applied after the inner_update operator has been applied. One interpretation of the MAML objective is that it is a differentiable estimate of a cross-validation loss with respect to a learner. Meta-training results in an inner_update that minimizes the cross-validation loss.
Suppose a task loss function $\mathcal{L}$ is defined with respect to model parameters $\theta$, input features $X$, output labels $Y$. Let $x_1, y_1$ and $x_2, y_2$ be identically distributed task instance data sampled from $X, Y$. Then MAML optimizes the following:
At meta-training time, the network learns to “quickly adapt” to x1, y1 in order to minimize cross-validation error on a new set of points x2. At deployment time (shown in the plot above), when we have a new task (new amplitude and phase not seen at training time), the model can apply the inner_update operator to fit the target sinusoid much faster and with fewer data samples than simply re-training the parameters with SGD.
@@ -235,7 +235,7 @@
Running this test gives us much more confidence in the correctness of the implementation! -Full source, including the code to restore the answer, can be found at https://github.com/matklad/dlx.
+Full source, including the code to restore the answer, can be found at https://github.com/matklad/dlx.It’s also interesting to reflect on the unusual effectiveness of linked list for this problem.
Remember that on the modern hardware, a Vec beats LinkedList for the overwhelming majority of the problems.
While linked lists have a better theoretical complexity for the insertion and removal from the middle, most benchmarks are dominated by the traversal time to get to this middle.
diff --git a/test/data/speedreader/rewriter/pages/issues_pages/nationalpost.com/distilled.html b/test/data/speedreader/rewriter/pages/issues_pages/nationalpost.com/distilled.html
index e67c8ba4a30e..26279052cfe4 100644
--- a/test/data/speedreader/rewriter/pages/issues_pages/nationalpost.com/distilled.html
+++ b/test/data/speedreader/rewriter/pages/issues_pages/nationalpost.com/distilled.html
@@ -3,5 +3,5 @@
https://smartcdn.gprod.postmedia.digital/nationalpost/wp-content/uploads/2020/06/abtracetogether.png?h=288&strip=all&quality=80 3x" width="96">
The Agency is planning to track population movement for roughly the next five years, including to address other public health issues, such as “other infectious diseases, chronic disease prevention and mental health,” the spokesperson added.
Privacy advocates raised concerns to the National Post about the long-term implications of the program.
“I think that the Canadian public will find out about many other such unauthorized surveillance initiatives before the pandemic is over—and afterwards,” David Lyon, author of Pandemic Surveillance and former director of the Surveillance Studies Centre at Queen’s University, said in an email.
Lyon warned that PHAC “uses the same kinds of ‘reassuring’ language as national security agencies use, for instance not mentioning possibilities for re-identifying data that has been ‘de-identified.’”
“In principle, of course, cell data can be used for tracking.”
Mobility data analysis “helps to advance public health objectives,” the PHAC spokesperson said. The findings have been regularly shared with provinces and territories via the special advisory committee to “inform public health messaging, planning and policy development,” the spokesperson said.
The data is also used for the COVID Trends portal, a dashboard that provides a summarized data of movement trends.
Lyon urged a need for greater information “regarding exactly what was done, what was achieved and whether or not it truly served the interests of Canadian citizens.”
Deploying surveillance tools for public health purposes also raises to the issue of equity, Martin French, an associate professor of Concordia University focusing on surveillance, privacy and social justice, noted in an email.
“There are populations that could experience an intensification of tracking that could have harmful (rather than beneficial) repercussions.”
Increased use of surveillance technology during the COVID-19 pandemic has created a new normal in the name of security, Lyon said.
“The pandemic has created opportunities for a massive surveillance surge on many levels—not only for public health, but also for monitoring those working, shopping and learning from home.”
“Evidence is coming in from many sources, from countries around the world, that what was seen as a huge surveillance surge—post 9/11—is now completely upstaged by pandemic surveillance,” he added.
In a notice posted earlier this week, the agency called for contractors with access to “cell-tower/operator location data in the response to the COVID-19 pandemic and for other public health applications.” It asks for “de-identified cell-tower based location data from across Canada” beginning from from Jan. 2019 until the end of the contract period on May 31, 2023, with possibility of three one-year extensions.
The contractor must provide anonymized data to PHAC and ensure its users have the ability to easily opt-out of mobility data sharing programs, the agency says.
PHAC’s privacy management division conducted an assessment and “determined that since no personal information is being acquired through this contract, there are no concerns under the Privacy Act,” the spokesperson said.
The Office of the Privacy Commissioner said it is “following up with PHAC to obtain more information about the proposed initiative” and could not provide additional comment at this time.