leigh/neogen
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

More docs (Readme#Adding-Languages)

Browse Source
This commit is contained in:
Daniel Mathiot
2021-08-30 21:34:29 +02:00
parent 5db8c287de
commit bd5ab9515f
3 changed files with 284 additions and 148 deletions
Download Patch File Download Diff File Expand all files Collapse all files

150
README.md
View File

@@ -173,154 +173,8 @@ There is a list of supported languages and fields, with their annotation style
## Adding Languages
### Configuration file
The configuration file for a language is in `lua/configurations/{lang}.lua`.
_Note: Be aware that Neogen uses Treesitter to operate. You can install [TSPlayground](https://github.com/nvim-treesitter/playground) to check the AST._
Below is a commented sample of the configuration file for `lua`.
```lua
-- Search for these nodes
parent = { "function", "local_function", "local_variable_declaration", "field", "variable_declaration" },
-- Traverse down these nodes and extract the information as necessary
data = {
-- If function or local_function is found as a parent
["function|local_function"] = {
-- Get second child from the parent node
["2"] = {
-- This second child has to be of type "parameters", otherwise does nothing
match = "parameters",
-- Extractor function that returns a set of TSname = values with values being of type string[]
extract = function(node)
local regular_params = neogen.utilities.extractors:extract_children_text("identifier")(node)
local varargs = neogen.utilities.extractors:extract_children_text("spread")(node)
return {
parameters = regular_params,
vararg = varargs,
}
end,
},
},
},
-- Custom lua locator that escapes from comments (More on locators below)
-- Passing nil will use the default locator
locator = require("neogen.locators.lua"),
-- Use default granulator and generator (More on them below)
granulator = nil,
generator = nil,
-- Template to use with the generator. (More on this below)
template = {
-- Which annotation convention to use
annotation_convention = "emmylua",
emmylua = {
{ nil, "- " },
{ "parameters", "- @param %s any" },
{ "vararg", "- @vararg any" },
{ "return_statement", "- @return any" }
}
},
```
The Neogen code is then divided in 3 major concepts:
### Locators
A locator tries to find (from the cursor node) one of the nodes from `parents` field specified in configuration.
This is the signature of the function:
```lua
function(node_info, nodes_to_match)
return node
end
```
- With `node_info` being a table with 2 fields:
```lua
{
root = root_node -- <TSnode>
current = current_node -- <TSnode>
}
```
- `nodes_to_match` is the field from `parents` in language configuration.
_Default: The default locator (in `lua/locators/default.lua`) just go back to the parent node of the current one and sees if it's one of the requested parents._
### Granulators
Now that a parent node is found (with locators) from the cursor location, it's time to use this node to find all requested fields.
The function signature is this:
```lua
function(parent_node, node_data)
return result
end
```
- `parent_node` being the node returned from the locator
- `result` is a table containing a set of `type = values` with values from type `string[]`, and type being a TS node name.
- `node_data` being the field `data` from configuration file. For example, if the `data` field is this one:
```lua
data = {
["function|local_function"] = {
-- get second child from the parent node
["2"] = {
-- it has to be of type "parameters"
match = "parameters",
extract = function(node)
local tree = {
{ retrieve = "all", node_type = "identifier", extract = true },
{ retrieve = "all", node_type = "spread", extract = true }
}
local nodes = neogen.utilities.nodes:matching_nodes_from(node, tree)
local res = neogen.utilities.extractors:extract_from_matched(nodes)
return {
parameters = res.identifier,
vararg = res.spread,
}
end,
},
},
}
```
Notes:
- If you create your own granulator, you can add any kind of parameters in the `data` field from configuration file as long as the function signature is the same provided.
- Utilities are provided. You can check out their documentation in `lua/utilities/`.
### Generators
A generator takes in the results from the granulator and tries to generate the template according to the language's configuration.
This is the function signature for a generator:
```lua
function(parent, data, template)
return start_row, start_col, generated_template
end
```
- `parent` is the parent node found with the locator
- `data` is the result from the granulator
- `template` being the `template` field from the language configuration file.
- `start_row` is the row in which we will append `generated_template`
- `start_col` is the col in which the `generated_template` will start
- `generated_template` is the output we will append on the specified locations.
1. Using the defaults to generate a new language support: [Adding Languages](./docs/adding-languages.md)
2. (advanced) Only if the defaults aren't enough, please see here: [Advanced Integration](./docs/advanced-integration.md)
## GIFS

132
docs/adding-languages.md
View File

@@ -32,3 +32,135 @@ func = { "function", "local_function", "local_variable_declaration", "field", "v
which will be `variable_declaration`. The parent node will now be this node.
### Granulator
### Concept
The granulator is a function that will try to fetch all desired information from a parent node.
### Default Granulator
The default granulator will use a data format like this one:
```lua
data = {
func = {
-- When the function is inside one of those
["local_variable_declaration|field|variable_declaration"] = {
["2"] = {
match = "function_definition",
extract = common_function_extractor,
},
},
-- When the function is in the root tree
["function_definition|function|local_function"] = {
["0"] = {
extract = common_function_extractor,
},
},
},
...
}
```
This means for our example, we found a parent node called `variable_declaration`. It'll fetch this data:
```lua
["local_variable_declaration|field|variable_declaration"] = {
["2"] = {
match = "function_definition",
extract = common_function_extractor,
},
},
```
This snippet means: From the parent node, try to execute the `extract` function from the second child called `function_definition`.
The extract function is important. It's here you'll parse the nodes and create a table corresponding the found content.
This is the extract function for lua functions:
```lua
local common_function_extractor = function(node)
local tree = {
{
retrieve = "first",
node_type = "parameters",
subtree = {
{ retrieve = "all", node_type = "identifier", extract = true },
{ retrieve = "all", node_type = "spread", extract = true },
},
},
{ retrieve = "first", node_type = "return_statement", extract = true },
}
local nodes = neogen.utilities.nodes:matching_nodes_from(node, tree)
local res = neogen.utilities.extractors:extract_from_matched(nodes)
return {
parameters = res.identifier,
vararg = res.spread,
return_statement = res.return_statement,
}
end
```
I exposed a function called `neogen.utilities.nodes:matching_nodes_from` which will try to generate a table of nodes following a tree
The tree in this example is very simple: from the node called `function_definition`, do:
- Retrieve the first node called `return_statement` and extract it
- From the first node called `parameters`, try to extract all nodes with name `identifier` and `spread`
The `neogen.utilities.extractors:extract_from_matched` takes the nodes table returned by the first function and will return a table containing the content of the nodes.
The granulator then return the table you want with the extracted values you want
### Generator
#### Concept
The generator will use the template provided in the language configuration and the extracted values from the granulator in order to create the annotations to be placed on the desired place.
#### Default Generator
The default generator uses a template format like this:
```lua
template = {
-- Which annotation convention to use
annotation_convention = "emmylua",
emmylua = {
{ nil, "- $1", { type = { "class", "func" } } }, -- add this string only on requested types
{ nil, "- $1", { no_results = true } }, -- Shows only when there's no results from the granulator
{ "parameters", "- @param %s $1|any" },
{ "vararg", "- @vararg $1|any" },
{ "return_statement", "- @return $1|any" },
{ "class_name", "- @class $1|any" },
{ "type", "- @type %s $1" },
},
ldoc = {
{ nil, "- $1", { no_results = true, type = { "func" } } },
{ nil, "- $1", { type = { "func" } } },
{ "parameters", " @tparam $1|any %s " },
{ "return_statement", " @treturn $1|any" },
},
},
```
The `annotation_convention` field will use the desired annotation template.
An annotation template is a set of tables, each one composed of 2 required params, and 1 optional table
- the name inside the granulator returned table
- the template to be formatted each time there is a value in the returned table from the granulator:
- Specifying `%s` will use the value from the table
- Adding `$1` will move the cursor there (with tab completion) if the user allowed it in the setup function
- the optional params:
- `no_results`: (bool, default false) will only use this field when there is no value returned by the granulator
- `type`: (string[], default nil) will only use this field for the required types. If not specified, will use this field for all types

150
docs/advanced-integration.md Normal file
View File

@@ -0,0 +1,150 @@
# Advanced Integration
## Configuration file
The configuration file for a language is in `lua/configurations/{lang}.lua`.
_Note: Be aware that Neogen uses Treesitter to operate. You can install [TSPlayground](https://github.com/nvim-treesitter/playground) to check the AST._
Below is a commented sample of the configuration file for `lua`.
```lua
-- Search for these nodes
parent = { "function", "local_function", "local_variable_declaration", "field", "variable_declaration" },
-- Traverse down these nodes and extract the information as necessary
data = {
-- If function or local_function is found as a parent
["function|local_function"] = {
-- Get second child from the parent node
["2"] = {
-- This second child has to be of type "parameters", otherwise does nothing
match = "parameters",
-- Extractor function that returns a set of TSname = values with values being of type string[]
extract = function(node)
local regular_params = neogen.utilities.extractors:extract_children_text("identifier")(node)
local varargs = neogen.utilities.extractors:extract_children_text("spread")(node)
return {
parameters = regular_params,
vararg = varargs,
}
end,
},
},
},
-- Custom lua locator that escapes from comments (More on locators below)
-- Passing nil will use the default locator
locator = require("neogen.locators.lua"),
-- Use default granulator and generator (More on them below)
granulator = nil,
generator = nil,
-- Template to use with the generator. (More on this below)
template = {
-- Which annotation convention to use
annotation_convention = "emmylua",
emmylua = {
{ nil, "- " },
{ "parameters", "- @param %s any" },
{ "vararg", "- @vararg any" },
{ "return_statement", "- @return any" }
}
},
```
The Neogen code is then divided in 3 major concepts:
## Locators
A locator tries to find (from the cursor node) one of the nodes from `parents` field specified in configuration.
This is the signature of the function:
```lua
function(node_info, nodes_to_match)
return node
end
```
- With `node_info` being a table with 2 fields:
```lua
{
root = root_node -- <TSnode>
current = current_node -- <TSnode>
}
```
- `nodes_to_match` is the field from `parents` in language configuration.
_Default: The default locator (in `lua/locators/default.lua`) just go back to the parent node of the current one and sees if it's one of the requested parents._
## Granulators
Now that a parent node is found (with locators) from the cursor location, it's time to use this node to find all requested fields.
The function signature is this:
```lua
function(parent_node, node_data)
return result
end
```
- `parent_node` being the node returned from the locator
- `result` is a table containing a set of `type = values` with values from type `string[]`, and type being a TS node name.
- `node_data` being the field `data` from configuration file. For example, if the `data` field is this one:
```lua
data = {
["function|local_function"] = {
-- get second child from the parent node
["2"] = {
-- it has to be of type "parameters"
match = "parameters",
extract = function(node)
local tree = {
{ retrieve = "all", node_type = "identifier", extract = true },
{ retrieve = "all", node_type = "spread", extract = true }
}
local nodes = neogen.utilities.nodes:matching_nodes_from(node, tree)
local res = neogen.utilities.extractors:extract_from_matched(nodes)
return {
parameters = res.identifier,
vararg = res.spread,
}
end,
},
},
}
```
Notes:
- If you create your own granulator, you can add any kind of parameters in the `data` field from configuration file as long as the function signature is the same provided.
- Utilities are provided. You can check out their documentation in `lua/utilities/`.
## Generators
A generator takes in the results from the granulator and tries to generate the template according to the language's configuration.
This is the function signature for a generator:
```lua
function(parent, data, template)
return start_row, start_col, generated_template
end
```
- `parent` is the parent node found with the locator
- `data` is the result from the granulator
- `template` being the `template` field from the language configuration file.
- `start_row` is the row in which we will append `generated_template`
- `start_col` is the col in which the `generated_template` will start
- `generated_template` is the output we will append on the specified locations.