APROPOS Summer School in Delft 27-29.06.2023

The second summer school of the APROPOS was co-organized by University of Amsterdam and hosted by TU Delft in the Netherlands. Contrary to previous events that have mostly consisted of presentations and lectures, this event invited the ESRs to participate more actively in discussions and breakout sessions with a particular focus on establishing collaborations. In this post, we summarize the topics raised during the three days.

A detailed schedule of the summer school is available in a previous blog post [1].

Figure 1 All the ESRs assembled outside TU Delft’s impressive library during a break from lectures and discussions.

1^st day (27 June)

The event started with an introduction to TU Delft and its campus environment, before progressing straight into the first three lectures.

“Simultaneous localization and mapping” by Dimitrios Stathis (KTH) and Saba Yousefzadeh (KTH)

This talk was split into multiple different parts, all revolving around robotic systems in production lines. In the first part, the presenter focused on SLAM algorithms [2] and its application for localizing maintenance robots. To improve accuracy and reduce uncertainty when making decisions, Atlas Copco fuse the data from IMU sensor with that of a camera. The fused data is processed by a Kalman filter: a three-step algorithm that, at discrete time stamps, predicts the current system state, then computes it based on measurements, and finally adjusts the model taking its prediction error into account. They demonstrated a functional system implemented on an AMD PYNQ-capable FPGA [3].

The second half of the talk discussed quantization. Quantization means systematically reducing the precision of numerical formats used in computing systems [4]. It is frequently used in ML applications, which have shown great resilience towards such approximations [5]. However, the presenter showed that these techniques may also find application in Kalman filters [6] used for pose estimation, for which many operations may be performed with half as many bits as in a baseline implementation with no noticeable accuracy degradation.

“Perception error modeling for autonomous driving” by Justin Dauwels (TU Delft)

Autonomous cars are gaining popularity, but intermittently one or more vehicles perceive their environment incorrectly and behave in unexpected ways or harm human beings [7, 8]. Unfortunately, current evaluation metrics fail to provide insights on the kinds of errors that these perception errors stem from [9]. This brings forth the question: “Can an autonomous vehicle make safe and robust decisions despite perception errors?”

Motivated by this, the presenter covered extensive work done in modeling vehicle perception systems and their resilience to a wide range of errors. As using real vehicles for such modeling purposes is cost-ineffective, evaluations are done in highly accurate simulations where the environment and different perception errors can be altered at will [9, 10]. The results thereof are plenty: a highlight being that vehicles that perceive the world through more than one sensor (e.g., both cameras and a LiDAR) show greater robustness to perception errors [10].

“Adaptive computation in DNNs” by Sam Leroux (Ghent University)

Existing DNN-based applications define static processing pipelines that dedicate the same amount of time and computational resources to all inputs. This is inefficient as some inputs are simpler to process than others. Adaptive computation exploits this to enable trading off accuracy for execution time [11]. The presenter covered three such techniques for adapting DNNs:

1. Early exits, which involves inserting classification layers at intermediate points in a DNN and only evaluating later layers if the classification confidence is too low.
2. Multiple branches, in which models are split horizontally into smaller, architecturally similar ones, whose outputs are combined in a weighted sum [11]. The weights are computed by another small DNN. Adaptation (and approximation) is introduced by skipping models whose weights are below a given threshold.
3. Iterative computing is applicable to models that are iterative by nature [12]. This feature pertains to ResNet [13] DNNs that integrate many architecturally identical sub-blocks with residual (pass-through) connections of inputs. In such networks, groups of identical blocks can be substituted by a loop over a single block till a given quality is reached.

Though illustrated for computer vision applications, these techniques can be applied to any DNN-based application. With all three techniques, one must carefully weigh potential savings against the overheads the techniques introduce.

At the end of the first day, the ESRs discussed resilience in work, a.k.a. work-life balance. Guided by the senior researchers and supervisors present, we openly talked about our common concerns. This led to important conclusions about not over-working, being able to say “no” and prioritize tasks when needed, and the importance of a life outside of work. Motivated in part by this discussion, the group spent the evening together on a scenic tour of Delft’s numerous canals.

2^nd day (28 June)

After a successful first day, the second day provided the ESRs with another three interesting lectures and two sessions for discussing leadership/supervision and collaborations.

“Approximation in healthcare wearables and fog gateways” by Anil Kanduri (University of Turku)

Healthcare wearables commonly combine data streams from multiple sensors to achieve better outcomes, or higher resilience or quality than attainable with a single sensor [14]. These data streams are highly heterogeneous in terms of data types, data volumes, energy consumption, resilience, and values, so processing them must be done intelligently. Moreover, being sensor-based, the data are often noisy. Applications typically deal with this noise through smart sensing, filtering, pre-processing, or model tuning, all of which by themselves represent approximation opportunities [15].

Within this space, there are many open research challenges, including:

1. Detecting noise at the sensor level.
2. Understanding the consequences of noise on transmission and compute.
3. Establishing a sense-compute action space and exploring their correlation.
4. Choosing and enforcing sense-compute co-optimization.

Co-optimizing sensing and compute requires awareness of both [16]. With such features available, applications may decide to rely on arbitrary subsets of their available data streams when it is beneficial for energy consumption or model accuracy.

“Governance of AI technologies” by Marijn Janssen (TU Delft)

Governance is a topic that we rarely consider when researching next-generation technologies, yet it is essential for ensuring that technology impacts our lives positively. It is a field encircled by terms such as norms, responsibilities, relationships, intervention, and therefrom trust [17]. The presenter outlined the challenges that recent technological advancements have brought, particularly in terms of assigning responsibilities to companies or individuals, and how these challenges are addressed both technologically and legally, from blockchain-based trust systems to international technology acts.

Despite the massive complexity of these challenges in the rapidly growing field of AI [18, 19], the presenter was optimistic, underlining their belief in national and international lawmakers. They also highlighted that lawmakers need researchers like us to guide them in understanding these technologies. One way to impact these processes is by taking part in the open discussions in local communities, national governments, or even at the international, EU level.

Following the second lecture, the ESRs and senior researchers present discussed experiences with supervision. Some ESRs have already (co-)supervised bachelor’s or master’s students, while others merely consider it a necessary skill for an academic career. Later, the discussion’s focus was directed to effective networking. The senior researchers highlighted the importance of interacting with other researchers, regardless of their titles, whose topics we find interesting at conferences. Following up on these initial encounters, however, is crucial to keep connections alive. Platforms like LinkedIn are great for this purpose.

“Open RAN as a key enabling framework for 5G and beyond networks” by Farinaz Kooshki (IS-Wireless)

The telecom business is notoriously difficult for new players to enter as it entails being compliant with many technical standards and competing with providers of closed, end-to-end systems. An open RAN, i.e., the protocols and architecture that enable any cellular device to connect with the network, is the key in enabling competition in this space through modularity. An open RAN permits advancements in programmability, interoperability, software, and thereby cost, most of which are driven by an increased number of vendors.

The speaker introduced the O-RAN Alliance [20], which is developing an open RAN standard. It is organized in eleven workgroups that each define part of the overall standard. Each part has open interfaces and is built as (virtualized) network functions. This permits service providers to purchase and combine individual functions from different vendors in a plug-and-play fashion. However, the open interfaces also introduce new security issues, which are yet to be resolved.

Before the end of the day, the ESRs discussed what constitutes good collaborations and how to foster more of them within the project. Everyone seemed interested in learning more about what the others are working on to identify potential overlaps. We agreed that collaborations must serve some greater purpose – they must be beneficial for all parties involved. Thankfully, as our topics are well-aligned, there are plenty of such opportunities to choose from. At the end of the session, we were asked to highlight a few of our colleagues with whom we were particularly interested in collaborating soon. These talks carried over to the evening, which we spent enjoying a wonderful dinner with all the school’s participants.

3^rd day (29 June)

The third and final day of the school began with a lecture before being dedicated wholly to discussions between the ESRs; first, the continuation of the collaboration discussions, and last, a panel debate on career paths after the PhD.

“DNN partitioning for IoT nodes” by Axel Jantsch (TU Wien)

IoT devices are numerous but often constrained by their size or battery capacity. In practice, these devices have roughly five orders of magnitude less compute and memory capacity than server nodes. This calls for drastic methods to reduce the compute requirements of DNNs executed on such devices. A way to do this, like the adaptation strategies presented by Sam Leroux, is by partitioning the DNN, performing its initial steps in the IoT device and the rest elsewhere [21, 22]. This technique may be applied to any pipelined application.

Identifying the optimal partitioning point with respect to latency or energy is complex. Communication typically dominates the overall energy consumption, though different protocols have different energy characteristics [22]. For high energy consumption protocols, the offloading sweet spot lies at the minimal amount of transmitted data, while for low energy consumption protocols, it lies at the extremes, i.e., the beginning or end of the processing pipeline. In both cases, the results are impacted by input size and the application of orthogonal techniques like quantization [4], pruning, and compression [23].

After the lecture, the ESRs were asked to group with one or two of the colleagues they had highlighted during the collaboration discussions the day before. In these groups, we were asked to concretize potential collaborations in more detail and make initial plans for how to best progress. Subsequently, we would present our plans in plenum. Judging by these presentations, this session was particularly productive as several new plans for joint work were announced; the results of which we will hopefully see published within the coming year!

The very final session was a panel debate about post-PhD life. The panel comprised: Aaron Ding (TU Delft), Axel Jantsch (TU Wien), Jari Nurmi (TAU), and Aleksandr Ometov (TAU), who have all chosen to follow academic career paths, though some with brief experience from industry. ESRs had the opportunity to ask about the motivation and reasons for this decision and experiences the panel members had gathered throughout their professional life. Highlights thereof include focusing on becoming independent as a researcher, being able to apply for funding, and estimating the skills and knowledge required in one, two, five, or even ten years into the future. The PhD is a great time to mature in these directions.

Acronyms:

• AxC: Approximate Computing
• AI: Artificial Intelligence
• DNN: Deep Neural Network
• IMU: Inertial Measurement Unit
• IoT: Internet of Things
• LiDAR: Light Detection and Ranging
• RAN: Radio Access Network
• SLAM: Simultaneous Localization and Mapping

Keywords:

• Approximate computing
• Artificial intelligence
• Edge computing
• Governance
• Internet of Things
• Machine learning
• Radio access network

References:

1. https://projects.tuni.fi/apropos/event/apropos-school-in-tu-delft/
2. https://en.wikipedia.org/wiki/Simultaneous_localization_and_mapping
3. https://www.xilinx.com/products/silicon-devices/soc.html
4. Nagel, Markus, et al. “A white paper on neural network quantization.” arXiv preprint arXiv:2106.08295(2021).
5. Yang, Jiwei, et al. “Quantization networks.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
6. Welch, Greg, and Gary Bishop. “An introduction to the Kalman filter.” (1995): 2.
7. https://www.washingtonpost.com/technology/2023/06/10/tesla-autopilot-crashes-elon-musk/
8. https://www.npr.org/2022/06/15/1105252793/nearly-400-car-crashes-in-11-months-involved-automated-tech-companies-tell-regul
9. Pandharipande, Ashish, et al. “Sensing and machine learning for automotive perception: A review.” IEEE Sensors Journal(2023).
10. Piazzoni, Andrea, et al. “On the Simulation of Perception Errors in Autonomous Vehicles.” arXiv preprint arXiv:2302.11919(2023).
11. Leroux, Sam, Bo Li, and Pieter Simoens. “Multi-branch neural networks for video anomaly detection in adverse lighting and weather conditions.” Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.
12. Leroux, Sam, et al. “Iamnn: Iterative and adaptive mobile neural network for efficient image classification.” arXiv preprint arXiv:1804.10123(2018).
13. He, Kaiming, et al. “Deep residual learning for image recognition.” Proceedings of the IEEE conference on computer vision and pattern recognition.
14. Naeini, Emad Kasaeyan, et al. “AMSER: Adaptive Multimodal Sensing for Energy Efficient and Resilient eHealth Systems.” 2022 Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, 2022.
15. Taufique, Zain, et al. “Approximate feature extraction for low power epileptic seizure prediction in wearable devices.” 2021 IEEE Nordic Circuits and Systems Conference (NorCAS). IEEE, 2021.
16. Shahhosseini, Sina, et al. “Towards Smart and Efficient Health Monitoring Using Edge-Enabled Situational-Awareness.” 2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS). IEEE, 2021.
17. Brous, Paul, and Marijn Janssen. “Trusted decision-making: Data governance for creating trust in data science decision outcomes.” Administrative Sciences4 (2020): 81.
18. Dwivedi, Yogesh K., et al. “Artificial Intelligence (AI): Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy.” International Journal of Information Management57 (2021): 101994.
19. Dwivedi, Yogesh K., et al. ““So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy.” International Journal of Information Management71 (2023): 102642.
20. https://www.o-ran.org/
21. Shahhosseini, Sina, et al. “Dynamic computation migration at the edge: Is there an optimal choice?.” Proceedings of the 2019 on Great Lakes Symposium on VLSI. 2019.
22. Shallari, Irida, et al. “Design space exploration for an IoT node: trade-offs in processing and communication.” IEEE Access9 (2021): 65078-65090.
23. Sanchez Leal, Isaac, et al. “Waist Tightening of CNNs: A Case study on Tiny YOLOv3 for Distributed IoT Implementations.” Proceedings of Cyber-Physical Systems and Internet of Things Week 2023. 241-246.