Invention Grant
- Patent Title: Universal lexical analyzers
-
Application No.: US15373204Application Date: 2016-12-08
-
Publication No.: US10169324B2Publication Date: 2019-01-01
- Inventor: Edo Nir , Albert Ribakovsky , Oren Gavriel , Gabi Shalev
- Applicant: HEWLETT PACKARD ENTERPRISE DEVELOPMENT LP
- Applicant Address: US CA Sunnyvale
- Assignee: ENTIT SOFTWARE LLC
- Current Assignee: ENTIT SOFTWARE LLC
- Current Assignee Address: US CA Sunnyvale
- Main IPC: G06F17/27
- IPC: G06F17/27

Abstract:
An example lexical analysis system may include lexical analyzer instructions and training instructions. The lexical analyzer instructions, when executed, may form a universal lexical analyzer that is to lexically analyze an input program by evaluating a function that maps an input tuple of lexemes to an output tuple of tokens based on input probability parameters. The universal lexical analyzer may be configurable for any programming language by inputting a set of probability parameters for the respective programming language as the input probability parameters of the function. The training instructions, when executed, may generate a set of probability parameters for any programming language by: receiving training programs that are in the respective programming language, each comprising a tuple of lexemes with associated tokens, and determining the set of probability parameters for the respective programming language based on lexemes and tokens in the training programs that are in the respective programming language.
Public/Granted literature
- US20180165273A1 UNIVERSAL LEXICAL ANALYZERS Public/Granted day:2018-06-14
Information query