Summary contributed by Connor Wright, who’s a 3rd year Philosophy student at the University of Exeter.
Link to original source + author at the bottom.
Mini-summary: With the world being increasingly governed by machine learning algorithms, how can we make sure we can trust in this process? Nicolas Papernotâs 33-minute video provides his and his groupâs findings on how to do just that. Ranging from topics of robustness, LP norms, differential privacy and deepfakes, the video focuses on two areas of making ML trustworthy. These are admission control at test time, and model governance. Considered at length, both are proposed as ways forward for helping make ML more trustworthy through their improvement on privacy, but they are not full-proof. A thought-provoking conclusion is then achieved to do with the alignment of human and ML norms.
Full summary:
This video provides Nicolas Papernotâs presentation of his involvement in a project on making ML more trustworthy. This included appeals to LP norms, differential privacy, admission control at test time, model governance, and deepfakes. These mentions will be dealt with as sections of the summary, ending with Papernotâs conclusions from his research.
Trustworthy ML:
In order to determine how to make ML more trustworthy, the group needed to determine what that would look like. To do this, they sought to include how a ML model could be robust against the threat of adversarial examples. Here, LP norms were utilised, placing the model as a constant predictor inside an LP ball, and thus making it less sensitive to perturbations. The LP norms then allowed for a new way of detecting adverse examples to be proposed, for they were able to be detected more clearly through their excessive exploitation of the excessive invariance that the LP norms had coded into the model. Hence, the question is asked whether such a method can be used to better detect threats through this resultant excessive exploitation, and then help train models to be robust against such threats in the future.
This is answered through framing the question in terms of an AI arms race. Traditional computer systems have treated cybersecurity in terms of âlocking up a houseâ. They âlock the doorâ in order to prevent intruders, and weigh this up against additionally âlocking the windowâ in case a bear was to break through. If this were to happen, the windows would then be locked, and weighed up against the possibility of a hawk descending through the chimney. If this were to happen, then the chimney would be locked etc. creating an AI arms race to try and defend against these threats. Instead, using LP norms to increase the robustness of the model to better detect these intruders, bears and hawks could be a way forward that saves time, money, and increases trust in ML.
Privacy:
One way to increase the trust using this model is in terms of privacy. ML lives on data, and thus it is sometimes at risk of the data subject wanting to privatise such data, even once itâs it has helped train the model. Thus, the group leant towards a definition of privacy as âdifferential privacyâ through their Private Aggregation of Teacher Ensembles (PATE) method. Here, instead of training the model on a whole data set, the group split the data up and assigned âteachersâ to each data partition. These partitions arenât related, so each teacher is trained independently to solve the same task of the model, whereby the group can aggregate the predictions made by the model, and then add some noise to the data in order to make the predictions more private. Hence, the data is only included in one of the data sets and only influences one of the teachers, meaning that if thereâs a prediction being made, your data is very unlikely to influence said prediction as you will only impact one of the predictions made. Resultantly, the data can be privatised more effectively through having less of an impact on the output, helping to align MLâs version of privacy to the human norms that fall under the same term.
The video then splits into two main topic areas: admission control at test time, and model governance, which I shall now introduce in turn.
Admission control at test time:
One way of abstaining from making a prediction within a model to help aid admission control is to promote a measure of uncertainty within the model, which the group linked back to the training data. The group then implemented the method of Deep K-Nearest Neighbours, which allowed them to open up the âblack boxâ of the model and see what was occurring in each layer (focusing on individual problems, rather than the model itself). Here, they looked at how each layer represents their test input, and in each of the layerâs representation space, they performed a nearest neighbour search. This is done until each layer is completed, whereby the nearest neighbour search will reveal how the labels of the nearest neighbour are inconsistent, and thus point to the problem.
One question then arises as to whether thereâs a possibility of defining objectives at the level of each layer to make models more amenable to test-time predictions. The group looked at this question in their use of âsoft nearest neighbour lossâ in 2019âs ICML conference. They asked whether it was better for the model to learn representation spaces that separate the data from classes with a large margin (like a support vector machine), or whether itâs better to entangle different classes together within the layers of representations.
They found that the latter was better for the Deep K-Nearest Neighbours method to estimate the uncertainty of the model. So, they then introduced the âsoft nearest-neighbour lossâ into the model to encourage it to entangle points from different classes in the layers of representations. The soft loss method will then encourage the model to co-opt features between different classes and the lower layers (which are able to be recognised using the same lower level features). This then helped them identify uncertainty when they have a test point that doesnât fall in any of the clusters, meaning they wouldâve had to have guessed whom their nearest neighbours are. Instead, thereâs now support when doing these searches as the different classes are subsequently entangled.
Model governance:
The group explored the problem of how to return data to a data subject that no longer wants their data to be utilised, despite it already being used to train the model, requiring a form of âmachine-unlearningâ. The question then becomes: is the differential privacy proposed enough to prevent such time-consuming machine-unlearning from occurring? As in, can the data being subjected to only affecting the outcome of one teacher be enough to satisfy what machine-unlearning would achieve?
The group decided that this probably wouldnât be the case. The modelâs data points (like in stochastic gradient descent) would still be influenced by the initial data point of the data subject, thus making it hard to remove their data entirely. Hence, Shared Isolated Sliced Aggregate training is then explored. Here, the method involves splitting the model into shards, and those shards into slices, meaning that the data point will only be in one shard and in one slice of that shard, whereby only one shard will need to be retrained rather than the whole model. Such retraining will then be quicker, for each shard completion contains a checkpoint before moving onto the next shard, proving a launching pad for retraining the model.
Deepfakes:
The notion of MLâs role in deepfakes is then considered, with progress in ML accelerating the progress in digital alteration. The group considered 3 approaches as how to combat this:
- Detect artifacts within the altered image (such as detecting imperfections, like imperfect body movements).
- Reveal content provenance (secure record of all entities and systems that manipulate a particular piece of content).
- Advocate a notion of total accountability (record every minute of your life).
The group believed that none of these 3 methods would achieve total coverage of all the problematic areas of deepfakes, so the supplementation of policy on areas such as predictive policing and feedback loops is required.
Conclusion:
The group concluded that research is needed in order to align ML with human norms. Once this is done, trustworthy ML is an opportunity to make ML better, and a cause that provides much food for thought for the future.
Original presentation by Nicolas Papernot et al.: https://youtu.be/UpGgIqLhaqoÂ