Lars Liden

Seattle, Washington, United States
1K followers 500+ connections

Join to view profile

About

I am a Principal Research Software Engineering Manager at Microsoft Research, where I…

Activity

Join now to see all activity

Experience & Education

  • Microsoft

View Lars’s full experience

See their title, tenure and more.

or

By clicking Continue to join or sign in, you agree to LinkedIn’s User Agreement, Privacy Policy, and Cookie Policy.

Licenses & Certifications

Publications

  • Knowledge-Grounded Dialogue Generation with a Unified Knowledge Representation

    Knowledge-grounded dialogue systems are challenging to build due to the lack of training data and heterogeneous knowledge sources. Existing systems perform poorly on unseen topics due to limited topics covered in the training data. In addition, heterogeneous knowledge sources make it challenging for systems to generalize to other tasks because knowledge sources in different knowledge representations require different knowledge encoders. To address these challenges, we present PLUG, a language…

    Knowledge-grounded dialogue systems are challenging to build due to the lack of training data and heterogeneous knowledge sources. Existing systems perform poorly on unseen topics due to limited topics covered in the training data. In addition, heterogeneous knowledge sources make it challenging for systems to generalize to other tasks because knowledge sources in different knowledge representations require different knowledge encoders. To address these challenges, we present PLUG, a language model that homogenizes different knowledge sources to a unified knowledge representation for knowledge-grounded dialogue generation tasks. PLUG is pre-trained on a dialogue generation task conditioned on a unified essential knowledge representation. It can generalize to different downstream knowledge-grounded dialogue generation tasks with a few training examples. The empirical evaluation on two benchmarks shows that our model generalizes well across different knowledge-grounded tasks. It can achieve comparable performance with state-of-the-art methods under a fully-supervised setting and significantly outperforms other methods in zero-shot and few-shot settings.

    See publication
  • SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teaching

    Transactions of the Association for Computational Linguistics

    We present a new method, Soloist,1 that uses transfer learning and machine teaching to build task bots at scale. We parameterize classical modular task-oriented dialog systems using a Transformer-based auto-regressive language model, which subsumes different dialog modules into a single neural model. We pre-train, on heterogeneous dialog corpora, a task-grounded response generation model, which can generate dialog responses grounded in user goals and real-world knowledge for task completion…

    We present a new method, Soloist,1 that uses transfer learning and machine teaching to build task bots at scale. We parameterize classical modular task-oriented dialog systems using a Transformer-based auto-regressive language model, which subsumes different dialog modules into a single neural model. We pre-train, on heterogeneous dialog corpora, a task-grounded response generation model, which can generate dialog responses grounded in user goals and real-world knowledge for task completion. The pre-trained model can be efficiently adapted to accomplish new tasks with a handful of task-specific dialogs via machine teaching, where training samples are generated by human teachers interacting with the system. Experiments show that (i)Soloist creates new state-of-the-art on well-studied task-oriented dialog benchmarks, including CamRest676 and MultiWOZ; (ii) in the few-shot fine-tuning settings, Soloist significantly outperforms existing methods; and (iii) the use of machine teaching substantially reduces the labeling cost of fine-tuning. The pre-trained models and codes are available at https://aka.ms/soloist.

    See publication
  • Datasets and Benchmarks for Task-Oriented Log Dialogue Ranking Task

    Interspeech 2020

    Although the data-driven approaches of some recent bot building platforms make it possible for a wide range of users to easily create dialogue systems, those platforms don’t offer tools for quickly identifying which log dialogues contain problems. Thus, in this paper, we (1) introduce a new task, log dialogue ranking, where the ranker places problematic dialogues higher (2) provide a collection of human-bot conversations in the restaurant inquiry task labelled with dialogue quality for ranker…

    Although the data-driven approaches of some recent bot building platforms make it possible for a wide range of users to easily create dialogue systems, those platforms don’t offer tools for quickly identifying which log dialogues contain problems. Thus, in this paper, we (1) introduce a new task, log dialogue ranking, where the ranker places problematic dialogues higher (2) provide a collection of human-bot conversations in the restaurant inquiry task labelled with dialogue quality for ranker training and evaluation (3) present a detailed description of the data collection pipeline, which is entirely based on crowd-sourcing (4) finally report a benchmark result of dialogue ranking, which shows the usability of the data and sets a baseline for future studies.

    See publication
  • Robust Conversational AI with Grounded Text Generation

    This article presents a hybrid approach based on a Grounded Text Generation (GTG) model to building robust task bots at scale. GTG is a hybrid model which uses a large-scale Transformer neural network as its backbone, combined with symbol-manipulation modules for knowledge base inference and prior knowledge encoding, to generate responses grounded in dialog belief state and real-world knowledge for task completion. GTG is pre-trained on large amounts of raw text and human conversational data…

    This article presents a hybrid approach based on a Grounded Text Generation (GTG) model to building robust task bots at scale. GTG is a hybrid model which uses a large-scale Transformer neural network as its backbone, combined with symbol-manipulation modules for knowledge base inference and prior knowledge encoding, to generate responses grounded in dialog belief state and real-world knowledge for task completion. GTG is pre-trained on large amounts of raw text and human conversational data, and can be fine-tuned to complete a wide range of tasks.
    The hybrid approach and its variants are being developed simultaneously by multiple research teams. The primary results reported on task-oriented dialog benchmarks are very promising, demonstrating the big potential of this approach. This article provides an overview of this progress and discusses related methods and technologies that can be incorporated for building robust conversational AI systems.

    See publication
  • Conversation Learner -- A Machine Teaching Tool for Building Dialog Managers for Task-Oriented Dialog Systems

    ACL Anthology

    Traditionally, industry solutions for building a task-oriented dialog system have relied on helping dialog authors define rule-based dialog managers, represented as dialog flows. While dialog flows are intuitively interpretable and good for simple scenarios, they fall short of performance in terms of the flexibility needed to handle complex dialogs. On the other hand, purely machine-learned models can handle complex dialogs, but they are considered to be black boxes and require large amounts of…

    Traditionally, industry solutions for building a task-oriented dialog system have relied on helping dialog authors define rule-based dialog managers, represented as dialog flows. While dialog flows are intuitively interpretable and good for simple scenarios, they fall short of performance in terms of the flexibility needed to handle complex dialogs. On the other hand, purely machine-learned models can handle complex dialogs, but they are considered to be black boxes and require large amounts of training data. In this demonstration, we showcase Conversation Learner, a machine teaching tool for building dialog managers. It combines the best of both approaches by enabling dialog authors to create a dialog flow using familiar tools, converting the dialog flow into a parametric model (e.g., neural networks), and allowing dialog authors to improve the dialog manager (i.e., the parametric model) over time by leveraging user-system dialog logs as training data through a machine teaching interface.

    See publication
  • Demonstration of interactive teaching for end-to-end dialog control with hybrid code networks

    Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue

    This is a demonstration of interactive teaching for practical end-to-end dialog systems driven by a recurrent neural network. In this approach, a developer teaches the network by interacting with the system and providing on-the-spot corrections. Once a system is deployed, a developer can also correct mistakes in logged dialogs. This demonstration shows both of these teaching methods applied to dialog systems in three domains: pizza ordering, restaurant information, and weather forecasts.

    See publication
Join now to see all publications

Patents

  • Hybrid transformer-based dialog processor

    Issued US-12032627-B2

    Systems and methods are provided for determining a response to a query in a dialog. An entity extractor extracts rules and conditions associated with the query and determines a particular task. The disclosed technology generates a transformer-based dialog embedding by pre-training a transformer using dialog corpora including a plurality of tasks. A task-specific classifier generates a first set of candidate responses based on rules and conditions associated with the task. The transformer-based…

    Systems and methods are provided for determining a response to a query in a dialog. An entity extractor extracts rules and conditions associated with the query and determines a particular task. The disclosed technology generates a transformer-based dialog embedding by pre-training a transformer using dialog corpora including a plurality of tasks. A task-specific classifier generates a first set of candidate responses based on rules and conditions associated with the task. The transformer-based dialog embedding generates a second set of candidate responses to the query. The classifier accommodates changes made to a task by an interactive dialog editor as machine teaching. A response generator generates a response based on the first and second sets of candidate responses using an optimization function. The disclosed technology leverages both a data-driven, generative model (a transformer) based on dialog corpora and a user-driven, task-specific rule-based classifier that accommodates updates in rules and conditions associated with a particular task

    See patent
  • Visualization of training dialogs for a conversational bot (Part 2)

    Issued US-11972331-B2

    This document relates to creating and/or updating a chatbot using a graphical user interface. For example, training dialogs for a chatbot can be displayed in a tree form on a graphical user interface. Based at least on interactions between a developer and the graphical user interface, the training dialogs can be modified in the tree form, and training dialogs can be updated based on the modifications provided on the tree form via the graphical user interface

    See patent
  • Training a user-system dialog in a task-oriented dialog system

    Issued US-11961509-B2

    Methods and systems are disclosed for improving dialog management for task-oriented machine teaching. The disclosed dialog builder leverages machine learning processing to improve development of dialog managers. In this way, the dialogue builder combines the strengths of both rule-based and machine-learned approaches to allow dialog authors to: (1) import a dialog graph developed using popular dialog compilers, (2) convert the developed graph into text-based training dialogs, (3) continuously…

    Methods and systems are disclosed for improving dialog management for task-oriented machine teaching. The disclosed dialog builder leverages machine learning processing to improve development of dialog managers. In this way, the dialogue builder combines the strengths of both rule-based and machine-learned approaches to allow dialog authors to: (1) import a dialog graph developed using popular dialog compilers, (2) convert the developed graph into text-based training dialogs, (3) continuously improve the trained dialogs based on log dialogs, and (4) generate a corrected dialog for retraining the machine learning.

    See patent
  • Computationally Customizing Instructional Content

    Issued 11,771,977

    A computing system causes instructional media to be played on a device to a user. An instructor in the instructional media provides guidance as to how to perform an activity when the instructional media is played on the device. The computing system obtains user data pertaining to performance of the activity by the user. The computing system generates a user-customized portion of the instructional media based upon the user data and a computer-implemented model. The computing system causes the…

    A computing system causes instructional media to be played on a device to a user. An instructor in the instructional media provides guidance as to how to perform an activity when the instructional media is played on the device. The computing system obtains user data pertaining to performance of the activity by the user. The computing system generates a user-customized portion of the instructional media based upon the user data and a computer-implemented model. The computing system causes the user-customized portion to be played on the device to the user, where the device emits audible words reproduced in a voice of the instructor, where the audible words are based upon the user data, and further where the device displays generated images of the instructor depicting the instructor speaking the audible words as the device emits the audible words.

    See patent
  • Visualization of training dialogs for a conversational bot (Part 1)

    Issued US-20230055720-A1

    This document relates to creating and/or updating a chatbot using a graphical user interface. For example, training dialogs for a chatbot can be displayed in a tree form on a graphical user interface. Based at least on interactions between a developer and the graphical user interface, the training dialogs can be modified in the tree form, and training dialogs can be updated based on the modifications provided on the tree form via the graphical user interface.

    See patent
  • Medium, system, and method for identifying collections associated with subjects appearing in a broadcast

    Issued US 10,405,059

    Disclosed are various embodiments for presenting collections of items related to subjects in a performance to viewers. The collections are assumed to be identified prior to the performance by individuals with knowledge of subjects expected to appear in the performance. The subject's appearance may be detected based on a user input indicating the appearance or by a subject recognition algorithm. The collections may be presented to a customer in a display also rendering the performance or in a…

    Disclosed are various embodiments for presenting collections of items related to subjects in a performance to viewers. The collections are assumed to be identified prior to the performance by individuals with knowledge of subjects expected to appear in the performance. The subject's appearance may be detected based on a user input indicating the appearance or by a subject recognition algorithm. The collections may be presented to a customer in a display also rendering the performance or in a separate display. The customer may select the collections and/or items therein for performing an action with the item, such as, for example purchasing the item.

    See patent
  • Assignation of emotional states to computer-implemented entities

    Issued US 10,402,718

    Described herein are various technologies pertaining to assigning an emotional state to a computer-implemented entity. The emotional state is a function of data values in a data stream output by a data source, such as a sensor or a web service. The data values in the data stream are processed over time to generate a measure value. Values that are indicative of a dynamic state of a measure relative to an optimal measure state are computed and assigned to the measure based upon the measure value.…

    Described herein are various technologies pertaining to assigning an emotional state to a computer-implemented entity. The emotional state is a function of data values in a data stream output by a data source, such as a sensor or a web service. The data values in the data stream are processed over time to generate a measure value. Values that are indicative of a dynamic state of a measure relative to an optimal measure state are computed and assigned to the measure based upon the measure value. A reaction of the computer-implemented entity is determined based upon these values. A customized personality for the computer-implemented entity maps reactions to respective emotional states, and an emotional state that is mapped to the reaction is assigned to the computer-implemented entity.

    See patent
  • Transaction bootstrapping to create relationships

    Issued US 10,339,549

    Techniques for providing friction-free transactions using geolocation and user identifiers are described herein. These techniques may ascertain a user's location based on a location of a mobile device. A transaction between the user and a merchant may be completed with zero or minimal input from the user based on the geolocation of the mobile device and the user identifiers. In some implementations, a transaction initiated earlier is completed when the mobile device arrives at the merchant…

    Techniques for providing friction-free transactions using geolocation and user identifiers are described herein. These techniques may ascertain a user's location based on a location of a mobile device. A transaction between the user and a merchant may be completed with zero or minimal input from the user based on the geolocation of the mobile device and the user identifiers. In some implementations, a transaction initiated earlier is completed when the mobile device arrives at the merchant. Additionally, a parent-child or similar relationship may be established between multiple devices. Security on the mobile device based may be provided by biometric identification and calculation of variance from regular movement patterns. Advertisements may be sent to the mobile device based on bids from merchants near to the mobile device. A relationship between the mobile device an a merchant may be bootstrapped from previous transactions that did not use the mobile device.

    Other inventors
    See patent
  • Cross-Mode Communication

    Issued US 10,171,410

    The techniques described herein are directed to cross-channel communication. The techniques can employ a central method, system, or mode ("Cross-Mode Manager" or "CMM" herein), which is enabled to send and/or receive messages over a multiplicity of modes. Users can enable the CMM through a variety of methods described herein on modes which the user uses to communicates. Once enabled on at least one channel which the user uses to communicate, the presence of the CMM across a multiplicity of…

    The techniques described herein are directed to cross-channel communication. The techniques can employ a central method, system, or mode ("Cross-Mode Manager" or "CMM" herein), which is enabled to send and/or receive messages over a multiplicity of modes. Users can enable the CMM through a variety of methods described herein on modes which the user uses to communicates. Once enabled on at least one channel which the user uses to communicate, the presence of the CMM across a multiplicity of modes allows a user to create and/or join a cross-channel communication session maintained by the CMM and to communicate between the user's communication mode and any of the multiplicity of modes for which the CMM has a presence. In other words, the techniques permit users to communicate across disparate channels from within a channel of their choosing.

    See patent
  • Budget Planner for Softlines #2

    Issued US 10,089,587

    Various approaches described herein enable various types of users, such as finance managers, vendors, and vendor managers to access up-to-date information in a retail environment that can assist with obtaining and tracking various items to be offered through that environment. A budget planner can analyze information such as historical performance, projection, and vendor data to establish a budget plan for various providers for a particular period. The budget information is fed to an assortment…

    Various approaches described herein enable various types of users, such as finance managers, vendors, and vendor managers to access up-to-date information in a retail environment that can assist with obtaining and tracking various items to be offered through that environment. A budget planner can analyze information such as historical performance, projection, and vendor data to establish a budget plan for various providers for a particular period. The budget information is fed to an assortment planner, which can use similar and other types of information to allocate that budget across various types and styles of item for each provider. A commitment tracker enables a user to commit to at least some portion of the allocation for various items, and track how well the orders are filled. Information at each level is fed back into the system to be almost instantly available and to adjust allocations for current or future periods.

    See patent
  • Relating items to objects detected in media

    Issued US 10,091,556

    Disclosed are various embodiments for relating items to objects detected in media. An item recognition application obtains media information associated with a broadcast being consumed form the client device. The item recognition application identifies objects in the media based on user input of the location of the object on the client device display and/or heuristic detection of the object. The user may indicate the object's location through an input device. The object may be heuristically…

    Disclosed are various embodiments for relating items to objects detected in media. An item recognition application obtains media information associated with a broadcast being consumed form the client device. The item recognition application identifies objects in the media based on user input of the location of the object on the client device display and/or heuristic detection of the object. The user may indicate the object's location through an input device. The object may be heuristically detected based on an algorithm that differentiates the object from the background. The item recognition application identifies items related to the object. The item recognition application then performs an action with the item.

    See patent
  • Creating connections and shared spaces

    Issued US 9,893,903

    Methods, computer systems, and computer-storage media are provided for connecting devices. Shared spaces may be created among one or more users and/or devices. Based on various factors including locations, histories among users, time of day, etc., spaces may be created among users such that content may be shared within the shared space. The spaces may be created using multiple technologies such that one user may connect to the space via a first identifier format while a second user may connect…

    Methods, computer systems, and computer-storage media are provided for connecting devices. Shared spaces may be created among one or more users and/or devices. Based on various factors including locations, histories among users, time of day, etc., spaces may be created among users such that content may be shared within the shared space. The spaces may be created using multiple technologies such that one user may connect to the space via a first identifier format while a second user may connect to the space via a second identifier format. Once created, content may be shared with one or more users of the space. Depending on privacy and/or security settings of the space, any user may invite other users to join the space. The content may be available to any participant once the space has terminated.

    Other inventors
    See patent
  • Transaction tracking and incentives

    Issued US 9,767,474

    Techniques for providing friction-free transactions using geolocation and user identifiers are described herein. These techniques may ascertain a user's location based on a location of a mobile device. A transaction between the user and a merchant may be completed with zero or minimal input from the user based on the geolocation of the mobile device and the user identifiers. In some implementations, a transaction initiated earlier is completed when the mobile device arrives at the merchant…

    Techniques for providing friction-free transactions using geolocation and user identifiers are described herein. These techniques may ascertain a user's location based on a location of a mobile device. A transaction between the user and a merchant may be completed with zero or minimal input from the user based on the geolocation of the mobile device and the user identifiers. In some implementations, a transaction initiated earlier is completed when the mobile device arrives at the merchant. Additionally, a parent-child or similar relationship may be established between multiple devices. Security on the mobile device based may be provided by biometric identification and calculation of variance from regular movement patterns. Advertisements may be sent to the mobile device based on bids from merchants near to the mobile device. Points may be accumulated through transactions with merchants and later redeemed for free or discounted goods and/or services.

    See patent
  • Associating Collections with Subjects

    Issued US 9,317,879

    Disclosed are various embodiments for presenting collections of items related to subjects in a performance to viewers. The collections are assumed to be identified prior to the performance by individuals with knowledge of subjects expected to appear in the performance. The subject's appearance may be detected based on a user input indicating the appearance or by a subject recognition algorithm. The collections may be presented to a customer in a display also rendering the performance or in a…

    Disclosed are various embodiments for presenting collections of items related to subjects in a performance to viewers. The collections are assumed to be identified prior to the performance by individuals with knowledge of subjects expected to appear in the performance. The subject's appearance may be detected based on a user input indicating the appearance or by a subject recognition algorithm. The collections may be presented to a customer in a display also rendering the performance or in a separate display. The customer may select the collections and/or items therein for performing an action with the item, such as, for example purchasing the item.

    See patent
  • Budget Planner For Softlines

    US 9,189,816

    Various approaches described herein enable various types of users, such as finance managers, vendors, and vendor managers to access up-to-date information in a retail environment that can assist with obtaining and tracking various items to be offered through that environment. A budget planner can analyze information such as historical performance, projection, and vendor data to establish a budget plan for various providers for a particular period. The budget information is fed to an assortment…

    Various approaches described herein enable various types of users, such as finance managers, vendors, and vendor managers to access up-to-date information in a retail environment that can assist with obtaining and tracking various items to be offered through that environment. A budget planner can analyze information such as historical performance, projection, and vendor data to establish a budget plan for various providers for a particular period. The budget information is fed to an assortment planner, which can use similar and other types of information to allocate that budget across various types and styles of item for each provider. A commitment tracker enables a user to commit to at least some portion of the allocation for various items, and track how well the orders are filled. Information at each level is fed back into the system to be almost instantly available and to adjust allocations for current or future periods.

    Other inventors
    See patent
  • Estimated Value Ratings for Electronic Messages

    US 9,231,897

    Estimated value ratings for electronic messages such as email may be generated by scraping the content of the email and comparing the scraped content to characteristics of a prospective recipient. The recipient may also provide an actual value rating which may be compared to the estimated value rating. The recipient may also provide a desired value rating which may be compared with the estimated value rating. More effective electronic message marketing may thereby be provided for both senders…

    Estimated value ratings for electronic messages such as email may be generated by scraping the content of the email and comparing the scraped content to characteristics of a prospective recipient. The recipient may also provide an actual value rating which may be compared to the estimated value rating. The recipient may also provide a desired value rating which may be compared with the estimated value rating. More effective electronic message marketing may thereby be provided for both senders and recipients.

    See patent
  • Outsourcing document-transformation tasks while protecting sensitive information

    US 9,390,282

    An outsourcing environment is described herein by which an outsourcing entity may delegate document-transformation tasks to at least one worker entity, while preventing the worker entity from gaining knowledge of sensitive items that may be contained within a non-obfuscated original document (NOD). In one example, the environment may transform the NOD into an obfuscated original document (OOD) by removing sensitive items from the NOD. The worker entity may perform formatting and/or other…

    An outsourcing environment is described herein by which an outsourcing entity may delegate document-transformation tasks to at least one worker entity, while preventing the worker entity from gaining knowledge of sensitive items that may be contained within a non-obfuscated original document (NOD). In one example, the environment may transform the NOD into an obfuscated original document (OOD) by removing sensitive items from the NOD. The worker entity may perform formatting and/or other document-transformation tasks on the OOD, without gaining knowledge of the sensitive items in the NOD, to produce an obfuscated transformed document (OTD). The environment may then allow for the outsourcing entity to view a content-restored version of the OTD.

    See patent
  • Tagging of Electronic Messages

    US 8,799,37

    An Email Service Provider (ESP) can generate tags for email messages that are received from senders, in addition to tags that are provided to the ESP by the senders. The ESP can send the email messages to selected potential recipients based on the tags that were provided by the senders and the tags that were generated by the ESP. Updating of selected tags by recipients may also be provided. The tags can be dynamic, derived and/or static tags.

    See patent
  • User Interest Tagging

    US 8,600,833

    Techniques and systems may be used to generate a list of interests of a user that interacts with a catalog of items, such as by purchasing the items from a host. The host may then generate a list of interests of the user using a taxonomy that is mapped to the catalog of items and the user interaction with the items. By generating the list of interests based on user interaction with the catalog, the list of interests may be generated based on factual data rather than user opinion. However, in…

    Techniques and systems may be used to generate a list of interests of a user that interacts with a catalog of items, such as by purchasing the items from a host. The host may then generate a list of interests of the user using a taxonomy that is mapped to the catalog of items and the user interaction with the items. By generating the list of interests based on user interaction with the catalog, the list of interests may be generated based on factual data rather than user opinion. However, in some instances, the user may additionally provide a self-rating for an interest that is identified by the host. In various aspects, the list of interests may be associated with a reviewer of items of the catalog. In some aspects, a reader may identify reviewers that have similar interests by comparing reviewers' interests to the reader's interests.

    See patent

Projects

Recommendations received

15 people have recommended Lars

Join now to view

More activity by Lars

View Lars’ full profile

  • See who you know in common
  • Get introduced
  • Contact Lars directly
Join to view full profile

Other similar profiles

Explore top content on LinkedIn

Find curated posts and insights for relevant topics all in one place.

View top content

Others named Lars Liden

Add new skills with these courses