fix(generator): more cleanup to plugin framework (#18027)

* fix(generator): more cleanup to plugin framework

* fix typo and package name

* add docs

* fix typo

* Update superset-frontend/webpack.config.js

Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com>

* fix generator reference

* add steps to tutorial and fix package version

* refine docs/readme

Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com>
This commit is contained in:
Ville Brofeldt 2022-01-17 10:23:35 +02:00 committed by GitHub
parent de8a1fa9c6
commit 8dea7f500b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
12 changed files with 300 additions and 290 deletions

View File

@ -0,0 +1,137 @@
---
name: Creating Visualization Plugins
menu: Contributing
route: /docs/contributing/creating-viz
index: 9
version: 1
---
## Creating Visualization Plugins
Visualizations in Superset are implemented in JavaScript or TypeScript. Superset
comes preinstalled with several visualizations types (hereafter "viz plugins") that
can be found under the `superset-frontend/plugins` directory. Viz plugins are added to
the application in the `superset-frontend/src/visualizations/presets/MainPreset.js`.
The Superset project is always happy to review proposals for new high quality viz
plugins. However, for highly custom viz types it is recommended to maintain a fork
of Superset, and add the custom built viz plugins by hand.
### Prerequisites
In order to create a new viz plugin, you need the following:
- Run MacOS or Linux (Windows is not officially supported, but may work)
- Node.js 16
- npm 7 or 8
A general familiarity with [React](https://reactjs.org/) and the npm/Node system is
also recommended.
### Creating a simple Hello World viz plugin
To get started, you need the Superset Yeoman Generator. It is recommended to use the
version of the template that ships with the version of Superset you are using. This
can be installed by doing the following:
```bash
npm i -g yo
cd superset-frontend/packages/generator-superset
npm i
npm link
```
After this you can proceed to create your viz plugin. Create a new directory for your
viz plugin with the prefix `superset-plugin-chart` and run the Yeoman generator:
```bash
mkdir /tmp/superset-plugin-chart-hello-world
cd /tmp/superset-plugin-chart-hello-world
```
Initialize the viz plugin:
```bash
yo @superset-ui/superset
```
After that the generator will ask a few questions (the defaults should be fine):
```
$ yo @superset-ui/superset
_-----_ ╭──────────────────────────╮
| | │ Welcome to the │
|--(o)--| │ generator-superset │
`---------´ │ generator! │
( _´U`_ ) ╰──────────────────────────╯
/___A___\ /
| ~ |
__'.___.'__
´ ` |° ´ Y `
? Package name: superset-plugin-chart-hello-world
? Description: Hello World
? What type of chart would you like? Time-series chart
create package.json
create .gitignore
create babel.config.js
create jest.config.js
create README.md
create tsconfig.json
create src/index.ts
create src/plugin/buildQuery.ts
create src/plugin/controlPanel.ts
create src/plugin/index.ts
create src/plugin/transformProps.ts
create src/types.ts
create src/SupersetPluginChartHelloWorld.tsx
create test/index.test.ts
create test/__mocks__/mockExportString.js
create test/plugin/buildQuery.test.ts
create test/plugin/transformProps.test.ts
create types/external.d.ts
create src/images/thumbnail.png
```
To build the viz plugin, run the following commands:
```
npm i --force
npm run build
```
Alternatively, to run the viz plugin in development mode (=rebuilding whenever changes
are made), start the dev server with the following command:
```
npm run dev
```
To add the package to Superset, go to the `superset-frontend` subdirectory in your
Superset source folder run
```bash
npm i -S /tmp/superset-plugin-chart-hello-world
```
If you publish your package to npm, you can naturally install directly from there, too.
After this edit the `superset-frontend/src/visualizations/presets/MainPreset.js`
and make the following changes:
```js
import { SupersetPluginChartHelloWorld } from 'superset-plugin-chart-hello-world';
```
to import the viz plugin and later add the following to the array that's passed to the
`plugins` property:
```js
new SupersetPluginChartHelloWorld().configure({ key: 'ext-hello-world' }),
```
After that the viz plugin should show up when you run Superset, e.g. the development
server:
```bash
npm run dev-server
```

View File

@ -1,217 +0,0 @@
---
name: Building Custom Viz Plugins
menu: Installation and Configuration
route: /docs/installation/building-custom-viz-plugins
index: 12
version: 1
---
This is a tutorial to help you build a "Hello World" viz plugin. The intent is to provide a basic
scaffolding to build any sort of data visualization, using any viz libary you'd like (e.g. ECharts,
AntV, HighCharts, VX, and D3.).
You can build the Hello World plugin by running a [Yeoman](https://yeoman.io/) generator, which
takes a few simple options, and provides this plugin scaffolding.
## Getting Set Up
### Install Yeoman and the Superset Package Generator
This Hello World plugin we'll be building is generated automatically with
[Yeoman](https://yeoman.io/). Let's first get that installed by opening up a terminal and installing
both the `yo` module and the
[superset package generator](https://github.com/apache-superset/superset-ui/tree/master/packages/generator-superset)
(`v0.14.7`) to create the new plugin.
```
npm install -g yo @superset-ui/generator-superset
```
### Install Superset
There are
[complete instructions](https://github.com/apache/superset#installation-and-configuration)
available on the [Superset Github repository](https://github.com/apache/superset). In a
nutshell, the easiest way is to:
1. Have a Mac or linux-based machine
2. Install [Docker](https://docs.docker.com/get-docker/)
3. Clone [the repository](https://github.com/apache/superset) to your computer
4. Use your terminal to `cd` into the `superset` directory
5. Run `docker-compose up`
6. Open _another_ terminal, and `cd` into `superset/superset-frontend`
7. Run `npm install` to load up all the npm packages.
8. Run `npm run dev-server` to spin up the Webpack hot-reloading server
9. Wait for it to build, and then open your browser to `http://localhost:9000` and log in with
`admin`/`admin`. You're off to the races! (Note: we'll be restarting this later)
### Install Superset-UI
1. Clone [the `superset-ui` repository](https://github.com/apache-superset/superset-ui) to your
computer. It can sit in the same parent directory as your `superset` repo
2. Use your terminal to `cd` into `superset-ui`
3. Run `yarn install` and wait for all the packages to get installed
## Build Your "Hello, World"
### ~~Write~~ _generate_ some code!
1. Using your terminal, `cd` into your local `superset-ui` repo folder and then into the `plugins`
subdirectory.
2. Make a new directory for your plugin, i.e. `mkdir plugin-chart-hello-world`. **Note:** we
_highly_ recommend following the `plugin-chart-your-plugin-name` pattern.
3. Now `cd plugin-chart-hello-world`
4. Finally, run `yo @superset-ui/superset`
5. Select `Create superset-ui chart plugin package` on the following screen:
<img src="/images/plugin-1-yeoman-select.png" />{' '}
6. Give it a name (in our case, go with the default, based on the folder name):
<img src="/images/plugin-2-yeoman-package-name.png" />
7. Give it a description (again, default is fine!)
<img src="/images/plugin-3-yeoman-description.png" />{' '}
8. Choose which type of React component you want to make (Class, or Function component).
<img src="/images/plugin-4-yeoman-component-type.png" />{' '}
9. Select whether you'd like your visualization to be timeseries-based or not
<img src="/images/plugin-5-yeoman-timeseries.png" />{' '}
10. Select whether or not you want to include badges at the top of your README file (really only
needed if you intend to contribute your plugin to the `superset-ui` repo).
<img src="/images/plugin-6-yeoman-badges.png" />{' '}
11. Admire all the files the generator has created for you. Note that EACH of these is chock full of
comments about what they're for, and how best to use them.
<img src="/images/plugin-7-yeoman-files.png" />{' '}
### Add your Plugin to Superset (with NPM Link)
Now, we want to see this thing actually RUN! To do that, we'll add your package to Superset and
embrace the magic power of `npm link` to see it in-situ, without needing to **build** the plugin, or
open any PRs on Github.
1. Add your package to the `package.json` file in `superset/superset-frontend`.
<img src="/images/plugin-8-package-json.png" />{' '}
Note: Do _not_ run `npm install`... explanation below.
2. Add your plugin to the `MainPreset.js` file (located in
`superset/superset-frontend/src/visualizations/presets/MainPreset.js`) in two places,
alongside the other plugins.
<img src="/images/plugin-9-mainpreset-import.png" />{' '}
{' '}
<img src="/images/plugin-9-mainpreset-register.png" />
3. Open a terminal window to `superset/superset-frontend`. If you did the Install Superset
steps above, you may still have webpack running there, and you can just stop it with `ctrol-c`.
If not, just open a new window and or `cd` to that directory path.
4) Use `npm link` to symlink plugin, using a relative path to `superset-ui` and your plugin folder,
e.g. `npm link ../../superset-ui/plugins/plugin-chart-hello-world`.
5. Restart your webpack dev server with `npm run dev-server`. You'll know it worked if you see a
line stating
`[Superset Plugin] Use symlink source for @superset-ui/plugin-chart-hello-world @ ^0.0.0`.
**NOTE:** If/when you do an `npm install` that erases the symlink generated by `npm link`, so you'll
have to redo those steps.
**NOTE:** Dynamic import is a work in progress. We hope you won't even need to DO this soon. We'll
be blogging again when that day comes, we assure you. In short, we have a goal to make editing
`package.json` and `MainPreset.js` unnecessary, so all the code changes are made in ONE repo.
### See it with your own eyes!
You should now be able to go to the Explore view in your local Superset and add a new chart! You'll
see your new plugin when you go to select your viz type.
<img src="/images/plugin-10-hello-thumbnail.png" />{' '}
Now you can load up some data, and you'll see it appear in the plugin!
<img src="/images/plugin-11-explore-view.png" />{' '}
The plugin also outputs three things to your browser's console:
- `formData`, a.k.a. everything sent into your viz from the controls
- `props`, as output from the `transformProps` file for your plugin's consumption
- The actual HTML element, which your plugin has hooks into for any necessary DOM maniupluation
<img src="/images/plugin-12-console-logs.png" />{' '}
## Make it Your Own
Now you're free to run wild with your new plugin! Here are a few places to start digging in:
### Read the comments and docs
Take a look through the full file tree of the plugin. The Readme gives details for the job of each
file. EACH of these files has been annotated with extensive comments of what the file is for, and
the basics of what you can do with it.
### Take control!
The plugin includes a couple of example controls, but you can certainly continue to add as many as
you need to. The comments/documentation within the controls file is a start, but we recommend
looking at existing `superset-ui` plugins for more examples of how you can implement controls to
enhance your queries, work with your data, and change your visualization's display.
### Build the perfect query
The `buildQuery` file where your plugin actually fetches data from the Superset backend. This file
builds he query "context" for your plugin. For a simple plugin, this file needn't do much. There are
a couple changes that need to be made for a timeseries plugin, thus the option in the Yeoman
generator.
This file also allows you to add various post-processing operations, to have the Superset backend
process your data in various ways (pivoting, etc), but that's a whole other topic we'll cover
separately in the near future.
### Style with Emotion
Each of these methods lets you add custom CSS styles using Emotion 👩‍🎤(a CSS-in-JS approach) which
has access to Superset's burgeoning set of theme variables, and also automatically scopes the styles
to your plugin, so they don't "leak" to other areas of Superset.
In the Hello World plugin, we've included a few example Theme variables (`colors`, `gridUnit`s, and
typographic weights/sizes). We'll be continuing to add more variables to this theme file as we
continue to push Superset (and the viz plugins) toward the standards of the Superset redesign (see
[SIP-34](https://github.com/apache/superset/issues/8976))
### Give it a thumbnail
Because come on... that's the fun part, right?
### Build it!
In this tutorial, you built your plugin in the `superset-ui` repo. This means you can use the
built-in build scripts that the repo provides. With your terminal of choice, simply `cd` into the
root directory of `supeset-ui` and run `yarn build`. This will kick off a build of ALL the Superset
plugins and packages, including yours.
### Test early, test often!
The Hello World plugin includes some basic Jest tests to act as a starting point to add unit tests
to your plugin. These do a quick sanity check that the plugin actually loads correctly, and then run
through the basics of making sure that your controls are properly respected by modifying the
resulting data and/or props of the plugin. Running `yarn test` from the root directory of
`superset-ui` will run all the tests for plugins/packages, including your Hello World.
### Deploying Custom Visualization to Production
To deploy plugins to a production environment, you must have additional code
inside Superset that includes the npm packages of your plugins so they can be installed in the frontend.
One option is to build your Dockerfile so it contains your custom visualization packages.

View File

@ -36,12 +36,11 @@ npm install -g @superset-ui/generator-superset
## Usage
Generate a new package or visualization plugin in `@superset-ui`
Generate a new package or visualization plugin:
```bash
cd superset-ui/packages
mkdir superset-ui-new-package
cd superset-ui-new-package
mkdir /tmp/superset-plugin-chart-hello-world
cd /tmp/superset-plugin-chart-hello-world
yo @superset-ui/superset
```

View File

@ -32,9 +32,7 @@ module.exports = class extends Generator {
name: 'packageName',
message: 'Package name:',
// Default to current folder name
default: _.kebabCase(
this.appname.replace('superset plugin chart', '').trim(),
),
default: _.kebabCase(this.appname),
},
{
type: 'input',
@ -77,6 +75,7 @@ module.exports = class extends Generator {
};
[
['gitignore.erb', '.gitignore'],
['babel.config.erb', 'babel.config.js'],
['jest.config.erb', 'jest.config.js'],
['package.erb', 'package.json'],

View File

@ -1,52 +1,40 @@
## superset-plugin-chart-<%= packageName %>
# <%= packageName %>
This plugin provides <%= description %> for Superset.
This is the <%= description %> Superset Chart Plugin.
### Usage
Configure `key`, which can be any `string`, and register the plugin. This `key` will be used to lookup this chart throughout the app.
To build the plugin, run the following commands:
```
npm i --force
npm run build
```
Alternatively, to run the plugin in development mode (=rebuilding whenever changes are made), start the dev server with the following command:
```
npm run dev
```
To add the package to Superset, go to the `superset-frontend` subdirectory in your Superset source folder (assuming both the `<%= packageName %>` plugin and `superset` repos are in the same root directory) and run
```
npm i -S ../../<%= packageName %>
```
After this edit the `superset-frontend/src/visualizations/presets/MainPreset.js` and make the following changes:
```js
import <%= packageLabel %>ChartPlugin from '@superset-ui/plugin-chart-<%= packageName %>';
new <%= packageLabel %>ChartPlugin()
.configure({ key: '<%= packageName %>' })
.register();
import { <%= packageLabel %> } from '<%= packageName %>';
```
Then use it via `SuperChart`. See [storybook](https://apache-superset.github.io/superset-ui/?selectedKind=plugin-chart-<%= packageName %>) for more details.
to import the plugin and later add the following to the array that's passed to the `plugins` property:
```js
<SuperChart
chartType="<%= packageName %>"
width={600}
height={600}
formData={...}
queriesData={[{
data: {...},
}]}
/>
new <%= packageLabel %>().configure({ key: '<%= packageName %>' }),
```
### File structure generated
After that the plugin should show up when you run Superset, e.g. the development server:
```
├── package.json
├── README.md
├── tsconfig.json
├── src
│   ├── <%= packageLabel %>.tsx
│   ├── images
│   │   └── thumbnail.png
│   ├── index.ts
│   ├── plugin
│   │   ├── buildQuery.ts
│   │   ├── controlPanel.ts
│   │   ├── index.ts
│   │   └── transformProps.ts
│   └── types.ts
├── test
│   └── index.test.ts
└── types
└── external.d.ts
npm run dev-server
```

View File

@ -0,0 +1,111 @@
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
lerna-debug.log*
# Diagnostic reports (https://nodejs.org/api/report.html)
report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json
# Runtime data
pids
*.pid
*.seed
*.pid.lock
# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov
# Coverage directory used by tools like istanbul
coverage
*.lcov
# nyc test coverage
.nyc_output
# Grunt intermediate storage (https://gruntjs.com/creating-plugins#storing-task-files)
.grunt
# Bower dependency directory (https://bower.io/)
bower_components
# node-waf configuration
.lock-wscript
# Compiled binary addons (https://nodejs.org/api/addons.html)
build/Release
# Dependency directories
node_modules/
jspm_packages/
# TypeScript v1 declaration files
typings/
# TypeScript cache
*.tsbuildinfo
# Optional npm cache directory
.npm
# Optional eslint cache
.eslintcache
# Microbundle cache
.rpt2_cache/
.rts2_cache_cjs/
.rts2_cache_es/
.rts2_cache_umd/
# Optional REPL history
.node_repl_history
# Output of 'npm pack'
*.tgz
# Yarn Integrity file
.yarn-integrity
# dotenv environment variables file
.env
.env.test
# parcel-bundler cache (https://parceljs.org/)
.cache
# Next.js build output
.next
# Nuxt.js build / generate output
.nuxt
dist
# Gatsby files
.cache/
# Comment in the public line in if your project uses Gatsby and *not* Next.js
# https://nextjs.org/blog/next-9-1#public-directory-support
# public
# vuepress build output
.vuepress/dist
# Serverless directories
.serverless/
# FuseBox cache
.fusebox/
# DynamoDB Local files
.dynamodb/
# TernJS port file
.tern-port
# IDEs
.idea/
# build
esm
lib

View File

@ -1,7 +1,7 @@
{
"name": "superset-plugin-chart-<%= packageName %>",
"name": "<%= packageName %>",
"version": "0.1.0",
"description": "Superset Chart - <%= description %>",
"description": "<%= description %>",
"sideEffects": false,
"main": "lib/index.js",
"module": "esm/index.js",
@ -9,38 +9,35 @@
"esm",
"lib"
],
"private": true,
"scripts": {
"build": "npm run build-cjs && npm run build-esm && npm run ts-types",
"build-cjs": "babel src --extensions \".ts,.tsx,.js,.jsx\" --copy-files --out-dir lib",
"build-clean": "npm run clean && npm run build",
"build-esm": "BABEL_OUTPUT=esm babel src --extensions \".ts,.tsx,.js,.jsx\" --copy-files --out-dir esm",
"clean": "rm -rf {lib,esm,tsconfig.tsbuildinfo}",
"dev": "webpack --mode=development --color --watch",
"dev": "BABEL_OUTPUT=esm babel src --extensions \".ts,.tsx,.js,.jsx\" --watch --copy-files --out-dir esm",
"prebuild": "rimraf {lib,esm,tsconfig.tsbuildinfo}",
"postbuild": "npm run test",
"ts-types": "tsc --build",
"test": "jest"
},
"author": "Superset",
"author": "My Name",
"license": "Apache-2.0",
"publishConfig": {
"access": "public"
},
"dependencies": {
},
"dependencies": {},
"peerDependencies": {
"react": "^16.13.1",
"@superset-ui/chart-controls": "*",
"@superset-ui/core": "*",
"@superset-ui/chart-controls": "*"
"react": "^16.13.1"
},
"devDependencies": {
"@airbnb/config-babel": "^2.0.1",
"@babel/cli": "^7.16.0",
"@types/jest": "^26.0.4",
"babel-loader": "^8.2.2",
"clean-webpack-plugin": "^4.0.0",
"jest": "^26.6.3",
"thread-loader": "^3.0.4",
"ts-loader": "^9.2.5",
"typescript": "^4.1.2",
"url-loader": "^4.1.1"
"rimraf": "^3.0.2",
"typescript": "^4.1.2"
}
}

View File

@ -17,7 +17,7 @@
* under the License.
*/
// eslint-disable-next-line import/prefer-default-export
export { default as <%= packageLabel %>ChartPlugin } from './plugin';
export { default as <%= packageLabel %> } from './plugin';
/**
* Note: this file exports the default export from <%= packageLabel %>.tsx.
* If you want to export multiple visualization modules, you will need to

View File

@ -22,7 +22,7 @@ import controlPanel from './controlPanel';
import transformProps from './transformProps';
import thumbnail from '../images/thumbnail.png';
export default class <%= packageLabel %>ChartPlugin extends ChartPlugin {
export default class <%= packageLabel %> extends ChartPlugin {
/**
* The constructor is used to pass relevant metadata and callbacks that get
* registered in respective registries that are used throughout the library

View File

@ -16,7 +16,7 @@
* specific language governing permissions and limitations
* under the License.
*/
import { <%= packageLabel %>ChartPlugin } from '../src';
import { <%= packageLabel %> } from '../src';
/**
* The example tests in this file act as a starting point, and
@ -26,8 +26,8 @@ import { <%= packageLabel %>ChartPlugin } from '../src';
* treated correctly (e.g. formData from plugin controls
* properly transform the data and/or any resulting props).
*/
describe('@superset-ui/plugin-chart-<%= packageName %>', () => {
describe('<%= packageName %>', () => {
it('exists', () => {
expect(<%= packageLabel %>ChartPlugin).toBeDefined();
expect(<%= packageLabel %>).toBeDefined();
});
});

View File

@ -40,6 +40,7 @@ test('generator-superset:plugin-chart:creates files', () =>
.withOptions({ skipInstall: true })
.then(function () {
assert.file([
'.gitignore',
'babel.config.js',
'jest.config.js',
'package.json',

View File

@ -285,7 +285,9 @@ const config = {
// resolve modules from `/superset_frontend/node_modules` and `/superset_frontend`
modules: ['node_modules', APP_DIR],
alias: {
// TODO: remove alias once React has been upgraaded to v. 17
// TODO: remove aliases once React has been upgraded to v. 17 and
// AntD version conflict has been resolved
antd: path.resolve(path.join(APP_DIR, './node_modules/antd')),
react: path.resolve(path.join(APP_DIR, './node_modules/react')),
},
extensions: ['.ts', '.tsx', '.js', '.jsx', '.yml'],
@ -432,13 +434,6 @@ Object.entries(packageConfig.dependencies).forEach(([pkg, relativeDir]) => {
const srcPath = path.join(APP_DIR, `./node_modules/${pkg}/src`);
const dir = relativeDir.replace('file:', '');
if (/^superset-plugin-/.test(pkg) && fs.existsSync(srcPath)) {
console.log(
`[Superset External Plugin] Use symlink source for ${pkg} @ ${dir}`,
);
// TODO: remove alias once React has been upgraaded to v. 17
config.resolve.alias[pkg] = path.resolve(APP_DIR, `${dir}/src`);
}
if (/^@superset-ui/.test(pkg) && fs.existsSync(srcPath)) {
console.log(`[Superset Plugin] Use symlink source for ${pkg} @ ${dir}`);
config.resolve.alias[pkg] = path.resolve(APP_DIR, `${dir}/src`);