Importing Flink into an IDE
#
将 Flink 导入集成开发环境 #
The sections below describe how to import the Flink project into an IDE
for the development of Flink itself. For writing Flink programs, please
refer to the Java API
and the Scala API
quickstart guides.
以下章节介绍了如何将 Flink 项目导入集成开发环境,以便开发 Flink 本身。关于编写 Flink 程序,请参阅 Java API 和 Scala API 快速入门指南。
Whenever something is not working in your IDE, try with the Maven command line first (mvn clean package -DskipTests
) as it might be your IDE that has a bug or is not properly set up.
如果 IDE 中出现故障,请先尝试使用 Maven 命令行(mvn clean package -DskipTests
),因为可能是 IDE 存在错误或设置不当。
Preparation
#
准备工作 #
To get started, please first checkout the Flink sources from one of our
repositories,
e.g.
要开始使用,请先从我们的资源库中选取 Flink 源,例如
git clone https://github.com/apache/flink.git
Ignoring Refactoring Commits
#
忽略重构提交 #
We keep a list of big refactoring commits in .git-blame-ignore-revs
. When looking at change
annotations using git blame
it’s helpful to ignore these. You can configure git and your IDE to
do so using:
我们在 .git-blame-ignore-revs
中保存了一份大的重构提交列表。当使用 git blame
查看变更注释时,忽略这些注释会很有帮助。您可以通过配置 git 和 IDE 来做到这一点:
git config blame.ignoreRevsFile .git-blame-ignore-revs
IntelliJ IDEA #
The following guide has been written for IntelliJ IDEA
2021.2. Some details might differ in other versions. Please make sure to follow all steps
accurately.
以下指南是针对 IntelliJ IDEA 2021.2 编写的。在其他版本中,某些细节可能会有所不同。请确保准确遵循所有步骤。
Importing Flink
#
导入 Flink #
- Choose “New” → “Project from Existing Sources”.
选择 "新建"→"来自现有资源的项目"。 - Select the root folder of the cloned Flink repository.
选择克隆 Flink 资源库的根文件夹。 - Choose “Import project from external model” and select “Maven”.
选择 "从外部模型导入项目 "并选择 "Maven"。 - Leave the default options and successively click “Next” until you reach the SDK section.
保留默认选项并依次点击 "下一步",直至到达 SDK 部分。 - If there is no SDK listed, create one using the “+” sign on the top left.
Select “JDK”, choose the JDK home directory and click “OK”.
Select the most suitable JDK version. NOTE: A good rule of thumb is to select
the JDK version matching the active Maven profile.
如果没有列出 SDK,请使用左上角的 "+"号创建一个。选择 "JDK",选择 JDK 主目录并单击 "OK"。选择最合适的 JDK 版本。注意:一个好的经验法则是选择与活动的 Maven 配置文件相匹配的 JDK 版本。 - Continue by clicking “Next” until the import is finished.
继续点击 "下一步",直到导入完成。 - Open the “Maven” tab (or right-click on the imported project and find “Maven”) and run
“Generate Sources and Update Folders”. Alternatively, you can run
mvn clean package -DskipTests
.
打开 "Maven "选项卡(或右键单击导入的项目并找到 "Maven"),然后运行 "生成源代码并更新文件夹"。或者,也可以运行mvn clean package -DskipTests
。 - Build the Project (“Build” → “Build Project”).
构建项目("构建"→"构建项目")。
Copyright Profile
#
版权简介 #
Every file needs to include the Apache license as a header. This can be automated in IntelliJ by
adding a Copyright profile:
每个文件都需要包含 Apache 许可证作为头文件。这可以通过在 IntelliJ 中添加版权配置文件来自动实现:
-
Go to “Settings” → “Editor” → “Copyright” → “Copyright Profiles”.
转到 "设置"→"编辑器"→"版权"→"版权配置文件"。 -
Add a new profile and name it “Apache”.
添加一个新的配置文件并命名为 "Apache"。 -
Add the following text as the license text:
添加以下文本作为许可证文本:Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
-
Go to “Editor” → “Copyright” and choose the “Apache” profile as the default profile for this project.
转到 "编辑器"→"版权",选择 "Apache "配置文件作为此项目的默认配置文件。 -
Click “Apply”. 点击 "应用"。
Required Plugins
#
所需插件 #
Go to “Settings” → “Plugins” and select the “Marketplace” tab. Search for the following plugins,
install them, and restart the IDE if prompted:
转到 "设置"→"插件",选择 "市场 "选项卡。搜索以下插件并安装,然后根据提示重启集成开发环境:
- Scala 斯卡拉
- Python – Required for PyFlink. If you do not
intend to work on PyFlink, you can skip this.
Python - PyFlink 需要。如果您不打算使用 PyFlink,可以跳过此选项。 - Save Actions 保存操作
- Checkstyle-IDEA
You will also need to install the google-java-format
plugin. However, a specific version of this plugin is required. Download
google-java-format v1.7.0.6
and install it as follows. Make sure to never update this plugin.
您还需要安装 google-java-format 插件。不过,需要该插件的特定版本。下载 google-java-format v1.7.0.6 并按如下步骤安装。确保永不更新此插件。
- Go to “Settings” → “Plugins”.
转到 "设置"→"插件"。 - Click the gear icon and select “Install Plugin from Disk”.
点击齿轮图标,选择 "从磁盘安装插件"。 - Navigate to the downloaded ZIP file and select it.
导航至下载的 ZIP 文件并选择它。
Code Formatting
#
代码格式化 #
Flink uses Spotless together with
google-java-format to format the Java code.
For Scala, it uses Spotless with scalafmt.
Flink 使用 Spotless 和 google-java-format 来格式化 Java 代码。对于 Scala,它使用 Spotless 和 scalafmt 来格式化 Java 代码。
It is recommended to automatically format your code by applying the following settings:
建议应用以下设置自动格式化代码:
- Go to “Settings” → “Other Settings” → “google-java-format Settings”.
转到 "设置"→"其他设置"→"google-java-format 设置"。 - Tick the checkbox to enable the plugin.
勾选复选框以启用插件。 - Change the code style to “Android Open Source Project (AOSP) style”.
将代码样式更改为 "Android 开放源代码项目 (AOSP) 样式"。 - Go to “Settings” → Editor → Code Style → Scala.
转到 "设置"→编辑器→代码风格→Scala。 - Change the “Formatter” to “scalafmt”.
将 "格式 "改为 "scalafmt"。 - Go to “Settings” → “Tools” → “Actions on Save”.
转到 "设置"→"工具"→"保存时的操作"。 - Under “Formatting Actions”, select “Optimize imports” and “Reformat code”.
在 "格式化操作 "下,选择 "优化导入 "和 "重新格式化代码"。 - From the “All file types list” next to “Reformat code”, select Java and Scala.
从 "重新格式化代码 "旁边的 "所有文件类型列表 "中,选择 Java 和 Scala。
For earlier IntelliJ IDEA versions:
对于早期的 IntelliJ IDEA 版本:
- Go to “Settings” → “Other Settings” → “Save Actions”.
转到 "设置"→"其他设置"→"保存操作"。 - Under “General”, enable your preferred settings for when to format the code, e.g. “Activate save actions on save”.
在 "常规 "下,就何时格式化代码启用您喜欢的设置,例如 "保存时激活保存操作"。 - Under “Formatting Actions”, select “Optimize imports” and “Reformat file”.
在 "格式化操作 "下,选择 "优化导入 "和 "重新格式化文件"。 - Under “File Path Inclusions”, add an entry for
.*\.java
and.*\.scala
to avoid formatting other file types.
在 "文件路径夹杂物 "下,为.*\.java
和.*\.scala
添加一个条目,以避免格式化其他文件类型。
You can also format the whole project (both Java and Scala) via Maven by using mvn spotless:apply
.
您还可以使用 mvn spotless:apply
,通过 Maven 格式化整个项目(包括 Java 和 Scala)。
Checkstyle For Java #
Checkstyle is used to enforce static coding guidelines.
Checkstyle 用于执行静态编码准则。
Some modules are not covered by Checkstyle, e.g. flink-core, flink-optimizer, and flink-runtime. Nevertheless, please make sure to conform to the checkstyle rules in these modules if you work in any of these modules.
有些模块不在 Checkstyle 的覆盖范围内,例如 flink-core、flink-optimizer 和 flink-runtime。不过,如果您在这些模块中工作,请务必遵守这些模块中的 Checkstyle 规则。
- Go to “Settings” → “Tools” → “Checkstyle”.
转到 "设置"→"工具"→"Checkstyle"。 - Set “Scan Scope” to “Only Java sources (including tests)”.
将 "扫描范围 "设置为 "仅 Java 源(包括测试)"。 - For “Checkstyle Version” select “9.3”.
在 "Checkstyle 版本 "中选择 "9.3"。 - Under “Configuration File” click the “+” icon to add a new configuration.
在 "配置文件 "下点击 "+"图标添加新配置。 - Set “Description” to “Flink”.
将 "描述 "设为 "闪烁"。 - Select “Use a local Checkstyle file” and point it to
tools/maven/checkstyle.xml
located within your cloned repository.
选择 "使用本地 Checkstyle 文件",并将其指向位于克隆版本库中的tools/maven/checkstyle.xml
文件。 - Select “Store relative to project location” and click “Next”.
选择 "相对于项目位置存储",然后点击 "下一步"。 - Configure the property
checkstyle.suppressions.file
with the valuesuppressions.xml
and click “Next”.
将属性checkstyle.suppressions.file
配置为suppressions.xml
并单击 "下一步"。 - Click “Finish”. 点击 "完成"。
- Select “Flink” as the only active configuration file and click “Apply”.
选择 "Flink "作为唯一激活的配置文件,然后点击 "应用"。
You can now import the Checkstyle configuration for the Java code formatter.
现在您可以导入用于 Java 代码格式化器的 Checkstyle 配置。
- Go to “Settings” → “Editor” → “Code Style” → “Java”.
转到 "设置"→"编辑器"→"代码样式"→"Java"。 - Click the gear icon next to “Scheme” and select “Import Scheme” → “Checkstyle Configuration”.
单击 "方案 "旁边的齿轮图标,然后选择 "导入方案"→"Checkstyle 配置"。 - Navigate to and select
tools/maven/checkstyle.xml
located within your cloned repository.
导航至并选择位于克隆版本库中的tools/maven/checkstyle.xml
。
To verify the setup, click “View” → “Tool Windows” → “Checkstyle” and find the “Check Module”
button in the opened tool window. It should report no violations.
要验证设置,请单击 "查看"→"工具窗口"→"Checkstyle",然后在打开的工具窗口中找到 "检查模块 "按钮。它应该不会报告违规情况。
Python for PyFlink #
Working on the flink-python module requires both a Java SDK and a Python SDK. However, IntelliJ IDEA
only supports one configured SDK per module. If you intend to work actively on PyFlink, it is
recommended to import the flink-python module as a separate project either in PyCharm
or IntelliJ IDEA for working with Python.
使用 flink-python 模块需要 Java SDK 和 Python SDK。然而,IntelliJ IDEA 只支持每个模块配置一个 SDK。如果您打算在 PyFlink 上积极开展工作,建议在 PyCharm 或 IntelliJ IDEA 中将 flink-python 模块作为一个单独的项目导入,以便与 Python 一起工作。
If you only occasionally need to work on flink-python and would like to get Python to work in
IntelliJ IDEA, e.g. to run Python tests, you can use the following guide.
如果您只是偶尔需要使用 flink-python,并希望让 Python 在 IntelliJ IDEA 中运行,例如运行 Python 测试,您可以使用以下指南。
- Follow Configure a virtual environment
to create a new Virtualenv Python SDK in your Flink project.
按照 配置虚拟环境 在 Flink 项目中创建一个新的 Virtualenv Python SDK。 - Find the flink-python module in the Project Explorer, right-click on it and choose
“Open Module Settings”. Alternatively, go to “Project Structure” → “Modules” and find the module
there.
在 "项目浏览器 "中找到 flink-python 模块,右击并选择 "打开模块设置"。或者,进入 "项目结构"→"模块",在那里找到该模块。 - Change “Module SDK” to the Virtualenv Python SDK you created earlier.
将 "模块 SDK "更改为之前创建的 Virtualenv Python SDK。 - Open the file
flink-python/setup.py
and install the dependencies when IntelliJ prompts you to do so.
打开flink-python/setup.py
文件,并根据 IntelliJ 的提示安装依赖项。
You can verify your setup by running some of the Python tests located in flink-python.
您可以运行 flink-python 中的一些 Python 测试来验证您的设置。
Common Problems
#
常见问题 #
This section lists issues that developers have run into in the past when working with IntelliJ.
本节列出了开发人员过去在使用 IntelliJ 时遇到的问题。
Compilation fails with invalid flag: --add-exports=java.base/sun.net.util=ALL-UNNAMED
#
编译失败,出现 invalid flag: --add-exports=java.base/sun.net.util=ALL-UNNAMED
#
This happens if the “java11” Maven profile is active, but an older JDK version is used. Go to
“View” → “Tool Windows” → “Maven” and uncheck the “java11” profile. Afterwards, reimport the
project.
如果 "java11 "Maven 配置文件处于活动状态,但使用的是较旧的 JDK 版本,就会出现这种情况。转到 "视图"→"工具窗口"→"Maven",取消选中 "java11 "配置文件。然后,重新导入项目。
Compilation fails with cannot find symbol: symbol: method defineClass(...) location: class sun.misc.Unsafe
#
编译失败,出现 cannot find symbol: symbol: method defineClass(...) location: class sun.misc.Unsafe
#
This happens if you are using JDK 11, but are working on a Flink version which doesn’t yet support
Java 11 (<= 1.9). Go to “Project Structure” → “Project Settings” → “Project” and select JDK 8 as
the Project SDK.
如果您使用的是 JDK 11,但 Flink 版本还不支持 Java 11 (<=1.9),就会出现这种情况。转到 "项目结构"→"项目设置"→"项目",选择 JDK 8 作为项目 SDK。
When switching back to newer Flink versions you may have to revert this change again.
当切换回更新的 Flink 版本时,您可能需要再次恢复这一更改。
Compilation fails with package sun.misc does not exist
#
编译失败,出现 package sun.misc does not exist
#
This happens if you are using JDK 11 and compile to Java 8 with the --release
option. This option is currently incompatible with our build setup.
Go to “Settings” → “Build, Execution, Deployment” → “Compiler” → “Java Compiler” and uncheck the “Use ‘–release’ option for cross-compilation (Java 9 and later)”.
如果您使用 JDK 11 并使用 --release
选项编译为 Java 8,则会出现这种情况。该选项目前与我们的编译设置不兼容。转到 "设置"→"编译、执行、部署"→"编译器"→"Java 编译器",取消选中 "交叉编译时使用'-release'选项(Java 9 及更高版本)"。
Examples fail with a NoClassDefFoundError
for Flink classes.
#
对于 Flink 类,示例失败时会出现 NoClassDefFoundError
错误。#
This happens if Flink dependencies are set to “provided”, resulting in them not being available
on the classpath. You can either check “Include dependencies with ‘Provided’ scope” in your
run configuration, or create a test that calls the main()
method of the example.
如果 Flink 依赖项被设置为 "已提供",导致它们在类路径上不可用,就会发生这种情况。您可以在运行配置中选中 "Include dependencies with 'Provided' scope"(使用 "提供 "作用域包含依赖项),或者创建一个调用 main()
示例方法的测试。
Eclipse #
Using Eclipse with Flink is currently not supported and discouraged. Please use
IntelliJ IDEA instead.
目前不支持并不鼓励在 Flink 中使用 Eclipse。请使用 IntelliJ IDEA 代替。
PyCharm #
If you intend to work on PyFlink, it is recommended to use
PyCharm as a separate IDE for the flink-python
module. The following guide has been written for 2019.1.3. Some details might differ in other
versions.
如果您打算在 PyFlink 上工作,建议使用 PyCharm 作为 flink-python 模块的独立 IDE。以下指南是针对 2019.1.3 编写的。在其他版本中,某些细节可能会有所不同。
Importing flink-python
#
导入 flink-python #
- Open the PyCharm IDE and choose (“File” →) “Open”.
打开 PyCharm IDE 并选择("文件"→)"打开"。 - Select the “flink-python” folder within your located repository.
选择所在版本库中的 "flink-python "文件夹。
Checkstyle For Python
#
用于 Python 的 Checkstyle #
Flake8 is used to enforce some coding guidelines.
Flake8 用于执行某些编码指南。
- Install flake8 for your Python interpreter using
pip install flake8
.
使用pip install flake8
为 Python 解释器安装 flake8。 - In PyCharm go to “Settings” → “Tools” → “External Tools”.
在 PyCharm 中进入 "设置"→"工具"→"外部工具"。 - Select the “+” button to add a new external tool.
选择 "+"按钮添加新的外部工具。 - Set “Name” to “flake8”. 将 "名称 "设为 "flake8"。
- Set “Description” to “Code Style Check”.
将 "描述 "设为 "代码样式检查"。 - Set “Program” to the path of your Python interpreter, e.g.
/usr/bin/python
.
将 "程序 "设置为 Python 解释器的路径,例如/usr/bin/python
. - Set “Arguments” to
-m flake8 --config=tox.ini
.
将 "参数 "设置为-m flake8 --config=tox.ini
。 - Set “Working Directory” to
$ProjectFileDir$
.
将 "工作目录 "设置为$ProjectFileDir$
。
You can verify the setup by right-clicking on any file or folder in the flink-python project
and running “External Tools” → “flake8”.
您可以右击 flink-python 项目中的任何文件或文件夹,然后运行 "外部工具"→"flake8 "来验证设置。