Publications
Results
- Showing results for:
- Reset all filters
Search results
-
Conference paperPardo F, Tavakoli A, Levdik V, et al., 2018,
Time limits in reinforcement learning
, International Conference on Machine Learning, Pages: 4042-4051In reinforcement learning, it is common to let anagent interact for a fixed amount of time with itsenvironment before resetting it and repeating theprocess in a series of episodes. The task that theagent has to learn can either be to maximize itsperformance over (i) that fixed period, or (ii) anindefinite period where time limits are only usedduring training to diversify experience. In thispaper, we provide a formal account for how timelimits could effectively be handled in each of thetwo cases and explain why not doing so can causestate-aliasing and invalidation of experience re-play, leading to suboptimal policies and traininginstability. In case (i), we argue that the termi-nations due to time limits are in fact part of theenvironment, and thus a notion of the remainingtime should be included as part of the agent’s in-put to avoid violation of the Markov property. Incase (ii), the time limits are not part of the envi-ronment and are only used to facilitate learning.We argue that this insight should be incorporatedby bootstrapping from the value of the state atthe end of each partial episode. For both cases,we illustrate empirically the significance of ourconsiderations in improving the performance andstability of existing reinforcement learning algo-rithms, showing state-of-the-art results on severalcontrol tasks.
-
Conference paperOlofsson S, Deisenroth M, Misener R, 2018,
Design of experiments for model discrimination hybridising analytical and data-driven approaches
, 35th International Conference on Machine Learning (ICML), Publisher: ICMLHealthcare companies must submit pharmaceuti-cal drugs or medical devices to regulatory bodiesbefore marketing new technology. Regulatorybodies frequently require transparent and inter-pretable computational modelling to justify a newhealthcare technology, but researchers may haveseveral competing models for a biological sys-tem and too little data to discriminate betweenthe models. In design of experiments for modeldiscrimination, the goal is to design maximallyinformative physical experiments in order to dis-criminate between rival predictive models. Priorwork has focused either on analytical approaches,which cannot manage all functions, or on data-driven approaches, which may have computa-tional difficulties or lack interpretable marginalpredictive distributions. We develop a method-ology introducing Gaussian process surrogatesin lieu of the original mechanistic models. Wethereby extend existing design and model discrim-ination methods developed for analytical modelsto cases of non-analytical models in a computa-tionally efficient manner.
-
Conference paperAltuncu MT, Mayer E, Yaliraki SN, et al., 2018,
From Text to Topics in Healthcare Records: An Unsupervised Graph Partitioning Methodology
, 2018 KDD Conference Proceedings - MLMH: Machine Learning for Medicine and HealthcareElectronic Healthcare Records contain large volumes of unstructured data,including extensive free text. Yet this source of detailed information oftenremains under-used because of a lack of methodologies to extract interpretablecontent in a timely manner. Here we apply network-theoretical tools to analysefree text in Hospital Patient Incident reports from the National HealthService, to find clusters of documents with similar content in an unsupervisedmanner at different levels of resolution. We combine deep neural networkparagraph vector text-embedding with multiscale Markov Stability communitydetection applied to a sparsified similarity graph of document vectors, andshowcase the approach on incident reports from Imperial College Healthcare NHSTrust, London. The multiscale community structure reveals different levels ofmeaning in the topics of the dataset, as shown by descriptive terms extractedfrom the clusters of records. We also compare a posteriori against hand-codedcategories assigned by healthcare personnel, and show that our approachoutperforms LDA-based models. Our content clusters exhibit good correspondencewith two levels of hand-coded categories, yet they also provide further medicaldetail in certain areas and reveal complementary descriptors of incidentsbeyond the external classification taxonomy.
-
Conference paperCully AHR, Demiris Y, 2018,
Hierarchical behavioral repertoires with unsupervised descriptors
, Genetic and Evolutionary Computation Conference 2018, Publisher: ACMEnabling artificial agents to automatically learn complex, versatile and high-performing behaviors is a long-lasting challenge. This paper presents a step in this direction with hierarchical behavioral repertoires that stack several behavioral repertoires to generate sophisticated behaviors. Each repertoire of this architecture uses the lower repertoires to create complex behaviors as sequences of simpler ones, while only the lowest repertoire directly controls the agent's movements. This paper also introduces a novel approach to automatically define behavioral descriptors thanks to an unsupervised neural network that organizes the produced high-level behaviors. The experiments show that the proposed architecture enables a robot to learn how to draw digits in an unsupervised manner after having learned to draw lines and arcs. Compared to traditional behavioral repertoires, the proposed architecture reduces the dimensionality of the optimization problems by orders of magnitude and provides behaviors with a twice better fitness. More importantly, it enables the transfer of knowledge between robots: a hierarchical repertoire evolved for a robotic arm to draw digits can be transferred to a humanoid robot by simply changing the lowest layer of the hierarchy. This enables the humanoid to draw digits although it has never been trained for this task.
-
Journal articleOlofsson S, Deisenroth MP, Misener R, 2018,
Design of Experiments for Model Discrimination using Gaussian Process Surrogate Models
, Computer Aided Chemical Engineering, Vol: 44, Pages: 847-852, ISSN: 1570-7946© 2018 Elsevier B.V. Given rival mathematical models and an initial experimental data set, optimal design of experiments for model discrimination discards inaccurate models. Model discrimination is fundamentally about finding out how systems work. Not knowing how a particular system works, or having several rivalling models to predict the behaviour of the system, makes controlling and optimising the system more difficult. The most common way to perform model discrimination is by maximising the pairwise squared difference between model predictions, weighted by measurement noise and model uncertainty resulting from uncertainty in the fitted model parameters. The model uncertainty for analytical model functions is computed using gradient information. We develop a novel method where we replace the black-box models with Gaussian process surrogate models. Using the surrogate models, we are able to approximately marginalise out the model parameters, yielding the model uncertainty. Results show the surrogate model method working for model discrimination for classical test instances.
-
Conference paperHurault G, Roekevisch E, Szegedi K, et al., 2018,
Predicting short- and long-term outcomes of a systemic therapy for atopic dermatitis using machine learning methods
, 10th George Rajka International Symposium on Atopic Dermatitis, Publisher: Wiley, Pages: E17-E18, ISSN: 1365-2133 -
Journal articleMuggleton S, Dai WZ, Sammut C, et al., 2018,
Meta-Interpretive Learning from noisy images
, Machine Learning, Vol: 107, Pages: 1097-1118, ISSN: 0885-6125Statistical machine learning is widely used in image classification. However, most techniques (1) require many images to achieve high accuracy and (2) do not provide support for reasoning below the level of classification, and so are unable to support secondary reasoning, such as the existence and position of light sources and other objects outside the image. This paper describes an Inductive Logic Programming approach called Logical Vision which overcomes some of these limitations. LV uses Meta-Interpretive Learning (MIL) combined with low-level extraction of high-contrast points sampled from the image to learn recursive logic programs describing the image. In published work LV was demonstrated capable of high-accuracy prediction of classes such as regular polygon from small numbers of images where Support Vector Machines and Convolutional Neural Networks gave near random predictions in some cases. LV has so far only been applied to noise-free, artificially generated images. This paper extends LV by (a) addressing classification noise using a new noise-telerant version of the MIL system Metagol, (b) addressing attribute noise using primitive-level statistical estimators to identify sub-objects in real images, (c) using a wider class of background models representing classical 2D shapes such as circles and ellipses, (d) providing richer learnable background knowledge in the form of a simple but generic recursive theory of light reflection. In our experiments we consider noisy images in both natural science settings and in a RoboCup competition setting. The natural science settings involve identification of the position of the light source in telescopic and microscopic images, while the RoboCup setting involves identification of the position of the ball. Our results indicate that with real images the new noise-robust version of LV using a single example (i.e. one-shot LV) converges to an accuracy at least comparable to a thirty-shot statistical machine learner on bot
-
Conference paperSaputra RP, Kormushev P, 2018,
ResQbot: a mobile rescue robot with immersive teleperception for casualty extraction
, Towards Autonomous Robotic Systems (TAROS) 2018, Publisher: Springer International Publishing AG, part of Springer Nature, Pages: 209-220, ISSN: 0302-9743In this work, we propose a novel mobile rescue robot equipped with an immersive stereoscopic teleperception and a teleoperation control. This robot is designed with the capability to perform safely a casualty-extraction procedure. We have built a proof-of-concept mobile rescue robot called ResQbot for the experimental platform. An approach called “loco-manipulation” is used to perform the casualty-extraction procedure using the platform. The performance of this robot is evaluated in terms of task accomplishment and safety by conducting a mock rescue experiment. We use a custom-made human-sized dummy that has been sensorised to be used as the casualty. In terms of safety, we observe several parameters during the experiment including impact force, acceleration, speed and displacement of the dummy’s head. We also compare the performance of the proposed immersive stereoscopic teleperception to conventional monocular teleperception. The results of the experiments show that the observed safety parameters are below key safety thresholds which could possibly lead to head or neck injuries. Moreover, the teleperception comparison results demonstrate an improvement in task-accomplishment performance when the operator is using the immersive teleperception.
-
Conference paperWang K, Shah A, Kormushev P, 2018,
SLIDER: a novel bipedal walking robot without knees
, Towards Autonomous Robotic Systems (TAROS) 2018, Publisher: Springer International Publishing AG, part of Springer Nature, Pages: 471-472, ISSN: 0302-9743In this work, we propose a novel mobile rescue robot equipped with an immersive stereoscopic teleperception and a teleoperation control. This robot is designed with the capability to perform safely a casualty-extraction procedure. We have built a proof-of-concept mobile rescue robot called ResQbot for the experimental platform. An approach called “loco-manipulation” is used to perform the casualty-extraction procedure using the platform. The performance of this robot is evaluated in terms of task accomplishment and safety by conducting a mock rescue experiment. We use a custom-made human-sized dummy that has been sensorised to be used as the casualty. In terms of safety, we observe several parameters during the experiment including impact force, acceleration, speed and displacement of the dummy’s head. We also compare the performance of the proposed immersive stereoscopic teleperception to conventional monocular teleperception. The results of the experiments show that the observed safety parameters are below key safety thresholds which could possibly lead to head or neck injuries. Moreover, the teleperception comparison results demonstrate an improvement in task-accomplishment performance when the operator is using the immersive teleperception.
-
Conference paperAltuncu T, Yaliraki SN, Barahona M, 2018,
Content-driven, unsupervised clustering of news articles through multiscale graph partitioning
, KDD 2018 - Workshop on Data Science Journalism and Media (DSJM)The explosion in the amount of news and journalistic content being generatedacross the globe, coupled with extended and instantaneous access to informationthrough online media, makes it difficult and time-consuming to monitor newsdevelopments and opinion formation in real time. There is an increasing needfor tools that can pre-process, analyse and classify raw text to extractinterpretable content; specifically, identifying topics and content-drivengroupings of articles. We present here such a methodology that brings togetherpowerful vector embeddings from Natural Language Processing with tools fromGraph Theory that exploit diffusive dynamics on graphs to reveal naturalpartitions across scales. Our framework uses a recent deep neural network textanalysis methodology (Doc2vec) to represent text in vector form and thenapplies a multi-scale community detection method (Markov Stability) topartition a similarity graph of document vectors. The method allows us toobtain clusters of documents with similar content, at different levels ofresolution, in an unsupervised manner. We showcase our approach with theanalysis of a corpus of 9,000 news articles published by Vox Media over oneyear. Our results show consistent groupings of documents according to contentwithout a priori assumptions about the number or type of clusters to be found.The multilevel clustering reveals a quasi-hierarchy of topics and subtopicswith increased intelligibility and improved topic coherence as compared toexternal taxonomy services and standard topic detection methods.
-
SoftwareCully A, Chatzilygeroudis K, Allocati F, et al., 2018,
Limbo: A Flexible High-performance Library for Gaussian Processes modeling and Data-Efficient Optimization
Limbo (LIbrary for Model-Based Optimization) is an open-source C++11 library for Gaussian Processes and data-efficient optimization (e.g., Bayesian optimization) that is designed to be both highly flexible and very fast. It can be used as a state-of-the-art optimization library or to experiment with novel algorithms with “plugin” components. Limbo is currently mostly used for data-efficient policy search in robot learning and online adaptation because computation time matters when using the low-power embedded computers of robots. For example, Limbo was the key library to develop a new algorithm that allows a legged robot to learn a new gait after a mechanical damage in about 10-15 trials (2 minutes), and a 4-DOF manipulator to learn neural networks policies for goal reaching in about 5 trials.The implementation of Limbo follows a policy-based design that leverages C++ templates: this allows it to be highly flexible without the cost induced by classic object-oriented designs (cost of virtual functions). The regression benchmarks show that the query time of Limbo’s Gaussian processes is several orders of magnitude better than the one of GPy (a state-of-the-art Python library for Gaussian processes) for a similar accuracy (the learning time highly depends on the optimization algorithm chosen to optimize the hyper-parameters). The black-box optimization benchmarks demonstrate that Limbo is about 2 times faster than BayesOpt (a C++ library for data-efficient optimization) for a similar accuracy and data-efficiency. In practice, changing one of the components of the algorithms in Limbo (e.g., changing the acquisition function) usually requires changing only a template definition in the source code. This design allows users to rapidly experiment and test new ideas while keeping the software as fast as specialized code.Limbo takes advantage of multi-core architectures to parallelize the internal optimization processes (optimization of the acquisition funct
-
Conference paperHurault G, Roekevisch E, Szegedi K, et al., 2018,
Development of computational tools to convert severity scores of atopic dermatitis for a probabilistic classification of symptom severity
, Annual Meeting of the British-Society-for-Investigative-Dermatology, Publisher: WILEY, Pages: E429-E429, ISSN: 0007-0963 -
Journal articleLaw M, Russo AM, Broda K, 2018,
The complexity and generality of learning answer set programs
, Artificial Intelligence, Vol: 259, Pages: 110-146, ISSN: 1872-7921Traditionally most of the work in the field of Inductive Logic Programming (ILP) has addressed the problem of learning Prolog programs. On the other hand, Answer Set Programming is increasingly being used as a powerful language for knowledge representation and reasoning, and is also gaining increasing attention in industry. Consequently, the research activity in ILP has widened to the area of Answer Set Programming, witnessing the proposal of several new learning frameworks that have extended ILP to learning answer set programs. In this paper, we investigate the theoretical properties of these existing frameworks for learning programs under the answer set semantics. Specifically, we present a detailed analysis of the computational complexity of each of these frameworks with respect to the two decision problems of deciding whether a hypothesis is a solution of a learning task and deciding whether a learning task has any solutions. We introduce a new notion of generality of a learning framework, which enables us to define a framework to be more general than another in terms of being able to distinguish one ASP hypothesis solution from a set of incorrect ASP programs. Based on this notion, we formally prove a generality relation over the set of existing frameworks for learning programs under answer set semantics. In particular, we show that our recently proposed framework, Context-dependent Learning from Ordered Answer Sets, is more general than brave induction, induction of stable models, and cautious induction, and maintains the same complexity as cautious induction, which has the highest complexity of these frameworks.
-
Conference paperBaroni P, Rago A, Toni F, 2018,
How many Properties do we need for Gradual Argumentation?
, AAAI 2018, Publisher: AAAIThe study of properties of gradual evaluation methods inargumentation has received increasing attention in recentyears, with studies devoted to various classes of frame-works/methods leading to conceptually similar but formallydistinct properties in different contexts. In this paper we pro-vide a systematic analysis for this research landscape by mak-ing three main contributions. First, we identify groups of con-ceptually related properties in the literature, which can be re-garded as based on common patterns and, using these pat-terns, we evidence that many further properties can be consid-ered. Then, we provide a simplifying and unifying perspec-tive for these properties by showing that they are all impliedby the parametric principles of (either strict or non-strict) bal-ance and monotonicity. Finally, we show that (instances of)these principles are satisfied by several quantitative argumen-tation formalisms in the literature, thus confirming their gen-eral validity and their utility to support a compact, yet com-prehensive, analysis of properties of gradual argumentation.
-
Conference paperKamthe S, Deisenroth MP, 2018,
Data-efficient reinforcement learning with probabilistic model predictive control
, Artificial Intelligence and Statistics, Publisher: PMLR, Pages: 1701-1710Trial-and-error based reinforcement learning(RL) has seen rapid advancements in recenttimes, especially with the advent of deep neural networks. However, the majority of autonomous RL algorithms require a large number of interactions with the environment. Alarge number of interactions may be impractical in many real-world applications, such asrobotics, and many practical systems have toobey limitations in the form of state spaceor control constraints. To reduce the numberof system interactions while simultaneouslyhandling constraints, we propose a modelbased RL framework based on probabilisticModel Predictive Control (MPC). In particular, we propose to learn a probabilistic transition model using Gaussian Processes (GPs)to incorporate model uncertainty into longterm predictions, thereby, reducing the impact of model errors. We then use MPC tofind a control sequence that minimises theexpected long-term cost. We provide theoretical guarantees for first-order optimality inthe GP-based transition models with deterministic approximate inference for long-termplanning. We demonstrate that our approachdoes not only achieve state-of-the-art dataefficiency, but also is a principled way for RLin constrained environments.
-
Journal articleCully AHR, Demiris Y, 2018,
Quality and diversity optimization: a unifying modular framework
, IEEE Transactions on Evolutionary Computation, Vol: 22, Pages: 245-259, ISSN: 1941-0026The optimization of functions to find the best solution according to one or several objectives has a central role in many engineering and research fields. Recently, a new family of optimization algorithms, named Quality-Diversity optimization, has been introduced, and contrasts with classic algorithms. Instead of searching for a single solution, Quality-Diversity algorithms are searching for a large collection of both diverse and high-performing solutions. The role of this collection is to cover the range of possible solution types as much as possible, and to contain the best solution for each type. The contribution of this paper is threefold. Firstly, we present a unifying framework of Quality-Diversity optimization algorithms that covers the two main algorithms of this family (Multi-dimensional Archive of Phenotypic Elites and the Novelty Search with Local Competition), and that highlights the large variety of variants that can be investigated within this family. Secondly, we propose algorithms with a new selection mechanism for Quality-Diversity algorithms that outperforms all the algorithms tested in this paper. Lastly, we present a new collection management that overcomes the erosion issues observed when using unstructured collections. These three contributions are supported by extensive experimental comparisons of Quality-Diversity algorithms on three different experimental scenarios.
-
Conference paperSaputra RP, Kormushev P, 2018,
ResQbot: A mobile rescue robot for casualty extraction
, 2018 ACM/IEEE International Conference on Human-Robot Interaction (HRI 2018), Publisher: Association for Computing Machinery, Pages: 239-240Performing search and rescue missions in disaster-struck environments is challenging. Despite the advances in the robotic search phase of the rescue missions, few works have been focused on the physical casualty extraction phase. In this work, we propose a mobile rescue robot that is capable of performing a safe casualty extraction routine. To perform this routine, this robot adopts a loco-manipulation approach. We have designed and built a mobile rescue robot platform called ResQbot as a proof of concept of the proposed system. We have conducted preliminary experiments using a sensorised human-sized dummy as a victim, to confirm that the platform is capable of performing a safe casualty extraction procedure.
-
Journal articleHerrero P, Bondia J, Giménez M, et al., 2018,
Automatic adaptation of Basal insulin using sensor-augmented pump therapy
, Journal of Diabetes Science and Technology, Vol: 12, Pages: 282-294, ISSN: 1932-2968BACKGROUND: People with insulin-dependent diabetes rely on an intensified insulin regimen. Despite several guidelines, they are usually impractical and fall short in achieving optimal glycemic outcomes. In this work, a novel technique for automatic adaptation of the basal insulin profile of people with diabetes on sensor-augmented pump therapy is presented. METHODS: The presented technique is based on a run-to-run control law that overcomes some of the limitations of previously proposed methods. To prove its validity, an in silico validation was performed. Finally, the artificial intelligence technique of case-based reasoning is proposed as a potential solution to deal with variability in basal insulin requirements. RESULTS: Over a period of 4 months, the proposed run-to-run control law successfully adapts the basal insulin profile of a virtual population (10 adults, 10 adolescents, and 10 children). In particular, average percentage time in target [70, 180] mg/dl was significantly improved over the evaluated period (first week versus last week): 70.9 ± 11.8 versus 91.1 ± 4.4 (adults), 46.5 ± 11.9 versus 80.1 ± 10.9 (adolescents), 49.4 ± 12.9 versus 73.7 ± 4.1 (children). Average percentage time in hypoglycemia (<70 mg/dl) was also significantly reduced: 9.7 ± 6.6 versus 0.9 ± 1.2 (adults), 10.5 ± 8.3 versus 0.83 ± 1.0 (adolescents), 10.9 ± 6.1 versus 3.2 ± 3.5 (children). When compared against an existing technique over the whole evaluated period, the presented approach achieved superior results on percentage of time in hypoglycemia: 3.9 ± 2.6 versus 2.6 ± 2.2 (adults), 2.9 ± 1.9 versus 2.0 ± 1.5 (adolescents), 4.6 ± 2.8 versus 3.5 ± 2.0 (children), without increasing the percentage time in hyperglycemia. CONCLUSION: The present study shows the potential of a novel technique to effectively adjust the basal insulin profile of a type 1 diab
-
Conference paperTavakoli A, Pardo F, Kormushev P, 2018,
Action branching architectures for deep reinforcement learning
, AAAI 2018, Publisher: AAAIDiscrete-action algorithms have been central to numerousrecent successes of deep reinforcement learning. However,applying these algorithms to high-dimensional action tasksrequires tackling the combinatorial increase of the numberof possible actions with the number of action dimensions.This problem is further exacerbated for continuous-actiontasks that require fine control of actions via discretization.In this paper, we propose a novel neural architecture fea-turing a shared decision module followed by several net-workbranches, one for each action dimension. This approachachieves a linear increase of the number of network outputswith the number of degrees of freedom by allowing a level ofindependence for each individual action dimension. To illus-trate the approach, we present a novel agent, called Branch-ing Dueling Q-Network (BDQ), as a branching variant ofthe Dueling Double Deep Q-Network (Dueling DDQN). Weevaluate the performance of our agent on a set of challeng-ing continuous control tasks. The empirical results show thatthe proposed agent scales gracefully to environments with in-creasing action dimensionality and indicate the significanceof the shared decision module in coordination of the dis-tributed action branches. Furthermore, we show that the pro-posed agent performs competitively against a state-of-the-art continuous control algorithm, Deep Deterministic PolicyGradient (DDPG).
-
Journal articleChamberlain B, Levy-Kramer J, Humby C, et al., 2018,
Real-time community detection in full social networks on a laptop
, PLoS ONE, Vol: 13, ISSN: 1932-6203For a broad range of research and practical applications it is important to understand the allegiances, communities and structure of key players in society. One promising direction towards extracting this information is to exploit the rich relational data in digital social networks (the social graph). As global social networks (e.g., Facebook and Twitter) are very large, most approaches make use of distributed computing systems for this purpose. Distributing graph processing requires solving many difficult engineering problems, which has lead some researchers to look at single-machine solutions that are faster and easier to maintain. In this article, we present an approach for analyzing full social networks on a standard laptop, allowing for interactive exploration of the communities in the locality of a set of user specified query vertices. The key idea is that the aggregate actions of large numbers of users can be compressed into a data structure that encapsulates the edge weights between vertices in a derived graph. Local communities can be constructed by selecting vertices that are connected to the query vertices with high edge weights in the derived graph. This compression is robust to noise and allows for interactive queries of local communities in real-time, which we define to be less than the average human reaction time of 0.25s. We achieve single-machine real-time performance by compressing the neighborhood of each vertex using minhash signatures and facilitate rapid queries through Locality Sensitive Hashing. These techniques reduce query times from hours using industrial desktop machines operating on the full graph to milliseconds on standard laptops. Our method allows exploration of strongly associated regions (i.e., communities) of large graphs in real-time on a laptop. It has been deployed in software that is actively used by social network analysts and offers another channel for media owners to monetize their data, helping them to continue to provide
This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.