{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 递归神经网络的简洁实现\n", ":label:`sec_rnn-concise`\n", "\n", "而：numref:`sec_rnn_scratch` 对于了解rnn是如何实现的很有指导意义，\n", "这既不方便也不快捷。\n", "本节将展示如何更有效地实现相同的语言模型\n", "使用高级API提供的函数\n", "一个深入学习的框架。\n", "我们像以前一样从读取时间机器数据集开始。\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%load ../utils/djl-imports\n", "%load ../utils/plot-utils\n", "%load ../utils/PlotUtils.java\n", "\n", "%load ../utils/Accumulator.java\n", "%load ../utils/Animator.java\n", "%load ../utils/Functions.java\n", "%load ../utils/StopWatch.java\n", "%load ../utils/Training.java\n", "%load ../utils/timemachine/Vocab.java\n", "%load ../utils/timemachine/RNNModelScratch.java\n", "%load ../utils/timemachine/TimeMachine.java" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import ai.djl.training.dataset.Record;" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "NDManager manager = NDManager.newBaseManager();" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 在DJL中创建数据集" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "在 DJL 中，处理数据集的理想而简洁的方法是使用内置的数据集，这些数据集可以轻松地围绕现有的 `Ndarray`，或者创建从 `RandomAccessDataset` 类扩展而来的自己的数据集。对于这一部分，我们将实现我们自己的。有关在 DJL 中创建自己的数据集的更多信息，请参阅：https://djl.ai/docs/development/how_to_use_dataset.html\n", "\n", "我们对 `TimeMachineDataset` 简洁地实现将替换先前创建的 `SeqDataLoader` 类。使用 DJL 格式的数据集，将允许我们使用已经内置的函数，这样我们就不必从头开始实现大多数功能。我们必须实现一个生成器、一个包含将数据保存到TimeMachineDataset对象的过程的prepare函数，以及一个get函数。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "public static class TimeMachineDataset extends RandomAccessDataset {\n", "\n", " private Vocab vocab;\n", " private NDArray data;\n", " private NDArray labels;\n", " private int numSteps;\n", " private int maxTokens;\n", " private int batchSize;\n", " private NDManager manager;\n", " private boolean prepared;\n", "\n", " public TimeMachineDataset(Builder builder) {\n", " super(builder);\n", " this.numSteps = builder.numSteps;\n", " this.maxTokens = builder.maxTokens;\n", " this.batchSize = builder.getSampler().getBatchSize();\n", " this.manager = builder.manager;\n", " this.data = this.manager.create(new Shape(0,35), DataType.INT32);\n", " this.labels = this.manager.create(new Shape(0,35), DataType.INT32);\n", " this.prepared = false;\n", " }\n", "\n", " @Override\n", " public Record get(NDManager manager, long index) throws IOException {\n", " NDArray X = data.get(new NDIndex(\"{}\", index));\n", " NDArray Y = labels.get(new NDIndex(\"{}\", index));\n", " return new Record(new NDList(X), new NDList(Y));\n", " }\n", "\n", " @Override\n", " protected long availableSize() {\n", " return data.getShape().get(0);\n", " }\n", "\n", " @Override\n", " public void prepare(Progress progress) throws IOException, TranslateException {\n", " if (prepared) {\n", " return;\n", " }\n", "\n", " Pair, Vocab> corpusVocabPair = null;\n", " try {\n", " corpusVocabPair = TimeMachine.loadCorpusTimeMachine(maxTokens);\n", " } catch (Exception e) {\n", " e.printStackTrace(); // 在tokenize()函数期间，异常可能来自未知的token类型。\n", " }\n", " List corpus = corpusVocabPair.getKey();\n", " this.vocab = corpusVocabPair.getValue();\n", "\n", " // 从一个随机偏移量（包括'numSteps-1'）开始到分区a\n", " // 序列\n", " int offset = new Random().nextInt(numSteps);\n", " int numTokens = ((int) ((corpus.size() - offset - 1) / batchSize)) * batchSize;\n", " NDArray Xs =\n", " manager.create(\n", " corpus.subList(offset, offset + numTokens).stream()\n", " .mapToInt(Integer::intValue)\n", " .toArray());\n", " NDArray Ys =\n", " manager.create(\n", " corpus.subList(offset + 1, offset + 1 + numTokens).stream()\n", " .mapToInt(Integer::intValue)\n", " .toArray());\n", " Xs = Xs.reshape(new Shape(batchSize, -1));\n", " Ys = Ys.reshape(new Shape(batchSize, -1));\n", " int numBatches = (int) Xs.getShape().get(1) / numSteps;\n", "\n", " NDList xNDList = new NDList();\n", " NDList yNDList = new NDList();\n", " for (int i = 0; i < numSteps * numBatches; i += numSteps) {\n", " NDArray X = Xs.get(new NDIndex(\":, {}:{}\", i, i + numSteps));\n", " NDArray Y = Ys.get(new NDIndex(\":, {}:{}\", i, i + numSteps));\n", " xNDList.add(X);\n", " yNDList.add(Y);\n", " }\n", " this.data = NDArrays.concat(xNDList);\n", " xNDList.close();\n", " this.labels = NDArrays.concat(yNDList);\n", " yNDList.close();\n", " this.prepared = true;\n", " }\n", "\n", " public Vocab getVocab() {\n", " return this.vocab;\n", " }\n", "\n", " public static final class Builder extends BaseBuilder {\n", " int numSteps;\n", " int maxTokens;\n", " NDManager manager;\n", "\n", " @Override\n", " protected Builder self() { return this; }\n", "\n", " public Builder setSteps(int steps) {\n", " this.numSteps = steps;\n", " return this;\n", " }\n", "\n", " public Builder setMaxTokens(int maxTokens) {\n", " this.maxTokens = maxTokens;\n", " return this;\n", " }\n", "\n", " public Builder setManager(NDManager manager) {\n", " this.manager = manager;\n", " return this;\n", " }\n", "\n", " public TimeMachineDataset build() throws IOException, TranslateException {\n", " TimeMachineDataset dataset = new TimeMachineDataset(this);\n", " return dataset;\n", " }\n", " }\n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "因此，我们将更新上一节中函数 `predictCh8`, `trainCh8`, `trainEpochCh8`, 和 `gradClipping` 的代码，以包含数据集逻辑，并允许函数从DJL接受 `AbstractBlock` 而不是只接受 `RNNModelScratch`." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "/** 在`prefix`后面生成新字符。 */\n", "public static String predictCh8(\n", " String prefix,\n", " int numPreds,\n", " Object net,\n", " Vocab vocab,\n", " Device device,\n", " NDManager manager) {\n", "\n", " List outputs = new ArrayList<>();\n", " outputs.add(vocab.getIdx(\"\" + prefix.charAt(0)));\n", " Functions.SimpleFunction getInput =\n", " () ->\n", " manager.create(outputs.get(outputs.size() - 1))\n", " .toDevice(device, false)\n", " .reshape(new Shape(1, 1));\n", "\n", " if (net instanceof RNNModelScratch) {\n", " RNNModelScratch castedNet = (RNNModelScratch) net;\n", " NDList state = castedNet.beginState(1, device);\n", "\n", " for (char c : prefix.substring(1).toCharArray()) { // 预热期\n", " state = (NDList) castedNet.forward(getInput.apply(), state).getValue();\n", " outputs.add(vocab.getIdx(\"\" + c));\n", " }\n", "\n", " NDArray y;\n", " for (int i = 0; i < numPreds; i++) {\n", " Pair pair = castedNet.forward(getInput.apply(), state);\n", " y = pair.getKey();\n", " state = pair.getValue();\n", "\n", " outputs.add((int) y.argMax(1).reshape(new Shape(1)).getLong(0L));\n", " }\n", " } else {\n", " AbstractBlock castedNet = (AbstractBlock) net;\n", " NDList state = null;\n", " for (char c : prefix.substring(1).toCharArray()) { // 预热期\n", " if (state == null) {\n", " // Begin state\n", " state =\n", " castedNet\n", " .forward(\n", " new ParameterStore(manager, false),\n", " new NDList(getInput.apply()),\n", " false)\n", " .subNDList(1);\n", " } else {\n", " state =\n", " castedNet\n", " .forward(\n", " new ParameterStore(manager, false),\n", " new NDList(getInput.apply()).addAll(state),\n", " false)\n", " .subNDList(1);\n", " }\n", " outputs.add(vocab.getIdx(\"\" + c));\n", " }\n", "\n", " NDArray y;\n", " for (int i = 0; i < numPreds; i++) {\n", " NDList pair =\n", " castedNet.forward(\n", " new ParameterStore(manager, false),\n", " new NDList(getInput.apply()).addAll(state),\n", " false);\n", " y = pair.get(0);\n", " state = pair.subNDList(1);\n", "\n", " outputs.add((int) y.argMax(1).reshape(new Shape(1)).getLong(0L));\n", " }\n", " }\n", "\n", " StringBuilder output = new StringBuilder();\n", " for (int i : outputs) {\n", " output.append(vocab.idxToToken.get(i));\n", " }\n", " return output.toString();\n", "}" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "/** 训练一个模型 */\n", "public static void trainCh8(\n", " Object net,\n", " RandomAccessDataset dataset,\n", " Vocab vocab,\n", " int lr,\n", " int numEpochs,\n", " Device device,\n", " boolean useRandomIter,\n", " NDManager manager)\n", " throws IOException, TranslateException {\n", " SoftmaxCrossEntropyLoss loss = new SoftmaxCrossEntropyLoss();\n", " Animator animator = new Animator();\n", "\n", " Functions.voidTwoFunction updater;\n", " if (net instanceof RNNModelScratch) {\n", " RNNModelScratch castedNet = (RNNModelScratch) net;\n", " updater =\n", " (batchSize, subManager) ->\n", " Training.sgd(castedNet.params, lr, batchSize, subManager);\n", " } else {\n", " // 已初始化网络\n", " AbstractBlock castedNet = (AbstractBlock) net;\n", " Model model = Model.newInstance(\"model\");\n", " model.setBlock(castedNet);\n", "\n", " Tracker lrt = Tracker.fixed(lr);\n", " Optimizer sgd = Optimizer.sgd().setLearningRateTracker(lrt).build();\n", "\n", " DefaultTrainingConfig config =\n", " new DefaultTrainingConfig(loss)\n", " .optOptimizer(sgd) // 优化器（损失函数）\n", " .optInitializer(\n", " new NormalInitializer(0.01f),\n", " Parameter.Type.WEIGHT) // 设置初始值设定项\n", " .optDevices(Engine.getInstance().getDevices(1)) // 设置所需的GPU数量\n", " .addEvaluator(new Accuracy()) // 模型精度\n", " .addTrainingListeners(TrainingListener.Defaults.logging()); // 日志\n", "\n", " Trainer trainer = model.newTrainer(config);\n", " updater = (batchSize, subManager) -> trainer.step();\n", " }\n", "\n", " Function predict =\n", " (prefix) -> predictCh8(prefix, 50, net, vocab, device, manager);\n", " // 训练和预测\n", " double ppl = 0.0;\n", " double speed = 0.0;\n", " for (int epoch = 0; epoch < numEpochs; epoch++) {\n", " Pair pair =\n", " trainEpochCh8(net, dataset, loss, updater, device, useRandomIter, manager);\n", " ppl = pair.getKey();\n", " speed = pair.getValue();\n", " if ((epoch + 1) % 10 == 0) {\n", " animator.add(epoch + 1, (float) ppl, \"ppl\");\n", " animator.show();\n", " }\n", " }\n", " System.out.format(\n", " \"perplexity: %.1f, %.1f tokens/sec on %s%n\", ppl, speed, device.toString());\n", " System.out.println(predict.apply(\"time traveller\"));\n", " System.out.println(predict.apply(\"traveller\"));\n", "}" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "/** 在一个epoch内训练一个模型 */\n", "public static Pair trainEpochCh8(\n", " Object net,\n", " RandomAccessDataset dataset,\n", " Loss loss,\n", " Functions.voidTwoFunction updater,\n", " Device device,\n", " boolean useRandomIter,\n", " NDManager manager)\n", " throws IOException, TranslateException {\n", " StopWatch watch = new StopWatch();\n", " watch.start();\n", " Accumulator metric = new Accumulator(2); // 训练损失总和，tokens\n", "\n", " try (NDManager childManager = manager.newSubManager()) {\n", " NDList state = null;\n", " for (Batch batch : dataset.getData(childManager)) {\n", " NDArray X = batch.getData().head().toDevice(device, true);\n", " NDArray Y = batch.getLabels().head().toDevice(device, true);\n", " if (state == null || useRandomIter) {\n", " // 在第一次迭代或\n", " // 使用随机抽样\n", " if (net instanceof RNNModelScratch) {\n", " state =\n", " ((RNNModelScratch) net)\n", " .beginState((int) X.getShape().getShape()[0], device);\n", " }\n", " } else {\n", " for (NDArray s : state) {\n", " s.stopGradient();\n", " }\n", " }\n", " if (state != null) {\n", " state.attach(childManager);\n", " }\n", "\n", " NDArray y = Y.transpose().reshape(new Shape(-1));\n", " X = X.toDevice(device, false);\n", " y = y.toDevice(device, false);\n", " try (GradientCollector gc = Engine.getInstance().newGradientCollector()) {\n", " NDArray yHat;\n", " if (net instanceof RNNModelScratch) {\n", " Pair pairResult = ((RNNModelScratch) net).forward(X, state);\n", " yHat = pairResult.getKey();\n", " state = pairResult.getValue();\n", " } else {\n", " NDList pairResult;\n", " if (state == null) {\n", " // 开始状态\n", " pairResult =\n", " ((AbstractBlock) net)\n", " .forward(\n", " new ParameterStore(manager, false),\n", " new NDList(X),\n", " true);\n", " } else {\n", " pairResult =\n", " ((AbstractBlock) net)\n", " .forward(\n", " new ParameterStore(manager, false),\n", " new NDList(X).addAll(state),\n", " true);\n", " }\n", " yHat = pairResult.get(0);\n", " state = pairResult.subNDList(1);\n", " }\n", "\n", " NDArray l = loss.evaluate(new NDList(y), new NDList(yHat)).mean();\n", " gc.backward(l);\n", " metric.add(new float[] {l.getFloat() * y.size(), y.size()});\n", " }\n", " gradClipping(net, 1, childManager);\n", " updater.apply(1, childManager); // 因为已经调用了“mean”函数\n", " }\n", " }\n", " return new Pair<>(Math.exp(metric.get(0) / metric.get(1)), metric.get(1) / watch.stop());\n", "}" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "/** 修剪梯度 */\n", "public static void gradClipping(Object net, int theta, NDManager manager) {\n", " double result = 0;\n", " NDList params;\n", " if (net instanceof RNNModelScratch) {\n", " params = ((RNNModelScratch) net).params;\n", " } else {\n", " params = new NDList();\n", " for (Pair pair : ((AbstractBlock) net).getParameters()) {\n", " params.add(pair.getValue().getArray());\n", " }\n", " }\n", " for (NDArray p : params) {\n", " NDArray gradient = p.getGradient().stopGradient();\n", " gradient.attach(manager);\n", " result += gradient.pow(2).sum().getFloat();\n", " }\n", " double norm = Math.sqrt(result);\n", " if (norm > theta) {\n", " for (NDArray param : params) {\n", " NDArray gradient = param.getGradient();\n", " gradient.muli(theta / norm);\n", " }\n", " }\n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "现在，我们将利用刚刚创建的数据集并分配所需的参数。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "int batchSize = 32;\n", "int numSteps = 35;\n", "\n", "TimeMachineDataset dataset = new TimeMachineDataset.Builder()\n", " .setManager(manager).setMaxTokens(10000).setSampling(batchSize, false)\n", " .setSteps(numSteps).build();\n", "dataset.prepare();\n", "Vocab vocab = dataset.getVocab();" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 定义模型\n", "\n", "高级API提供递归神经网络的实现。\n", "我们构造了一个包含单个隐层和256个隐单元的递归神经网络层`rnn_layer`。\n", "事实上，我们甚至还没有讨论多层的含义——这将发生在 :numref:`sec_deep_rnn`.\n", "现在，只需说多个层相当于一个RNN层的输出被用作下一个 RNN 层的输入即可。\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "int numHiddens = 256;\n", "RNN rnnLayer = RNN.builder().setNumLayers(1)\n", " .setStateSize(numHiddens).optReturnState(true).optBatchFirst(false).build();" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "初始化隐藏状态很简单。\n", "我们调用成员函数 `beginState` (在DJL中，我们不必在以后第一次运行 `forward` 时运行 `beginState` 来指定结果状态，因为在我们第一次运行`forward`, 时，DJL会运行此逻辑，但出于演示目的，我们将在此处创建此逻辑)。\n", "这将返回一个列表 (`state`)\n", "包含\n", "初始隐藏状态\n", "对于minibatch中的每个示例，\n", "是谁的形状\n", "（隐藏层的数量、批次大小、隐藏单元的数量）。\n", "对于某些型号\n", "待稍后介绍\n", "(例如，长-短期记忆),\n", "这样一份名单也不例外\n", "包含其他信息。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "public static NDList beginState(int batchSize, int numLayers, int numHiddens) {\n", " return new NDList(manager.zeros(new Shape(numLayers, batchSize, numHiddens)));\n", "}\n", "\n", "NDList state = beginState(batchSize, 1, numHiddens);\n", "System.out.println(state.size());\n", "System.out.println(state.get(0).getShape());" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "具有隐藏状态和输入，\n", "我们可以用\n", "更新后的隐藏状态。\n", "应该强调的是\n", "the \"output\" (`Y`) of `rnnLayer`\n", "*不* 涉及输出层的计算：\n", "指\n", "*每个* 时间步骤的隐藏状态，\n", "它们可以作为输入\n", "到后续的输出层。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "此外，\n", "(`stateNew`) 返回的更新的隐藏状态 `rnnLayer`\n", "指隐藏状态\n", "在小批量的*最后*时间步。\n", "它可用于初始化\n", "一个epoch下一个小批量的隐藏状态\n", "在顺序分区中。\n", "对于多个隐藏层，\n", "将存储每个层的隐藏状态\n", "在这个变量(`stateNew`)中。\n", "对于某些型号\n", "待稍后介绍\n", "(例如，长-短期记忆),\n", "这个变量也是\n", "包含其他信息。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "NDArray X = manager.randomUniform (0, 1,new Shape(numSteps, batchSize, vocab.length()));\n", "\n", "NDList input = new NDList(X, state.get(0));\n", "rnnLayer.initialize(manager, DataType.FLOAT32, input.getShapes());\n", "NDList forwardOutput = rnnLayer.forward(new ParameterStore(manager, false), input, false);\n", "NDArray Y = forwardOutput.get(0);\n", "NDArray stateNew = forwardOutput.get(1);\n", "\n", "System.out.println(Y.getShape());\n", "System.out.println(stateNew.getShape());" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "类似于 :numref:`sec_rnn_scratch`,\n", "我们定义了一个 `RNNModel` 类 \n", "对于完整的RNN模型。\n", "注意 `rnnLayer` 只包含隐藏的重复层，我们需要创建一个单独的输出层。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "public class RNNModel extends AbstractBlock {\n", "\n", " private RNN rnnLayer;\n", " private Linear dense;\n", " private int vocabSize;\n", "\n", " public RNNModel(RNN rnnLayer, int vocabSize) {\n", " this.rnnLayer = rnnLayer;\n", " this.addChildBlock(\"rnn\", rnnLayer);\n", " this.vocabSize = vocabSize;\n", " this.dense = Linear.builder().setUnits(vocabSize).build();\n", " this.addChildBlock(\"linear\", dense);\n", " }\n", "\n", " @Override\n", " protected NDList forwardInternal(ParameterStore parameterStore, NDList inputs, boolean training, PairList params) {\n", " NDArray X = inputs.get(0).transpose().oneHot(vocabSize);\n", " inputs.set(0, X);\n", " NDList result = rnnLayer.forward(parameterStore, inputs, training);\n", " NDArray Y = result.get(0);\n", " NDArray state = result.get(1);\n", "\n", " int shapeLength = Y.getShape().dimension();\n", " NDList output = dense.forward(parameterStore, new NDList(Y\n", " .reshape(new Shape(-1, Y.getShape().get(shapeLength-1)))), training);\n", " return new NDList(output.get(0), state);\n", " }\n", " \n", " @Override\n", " public void initializeChildBlocks(NDManager manager, DataType dataType, Shape... inputShapes) {\n", " Shape shape = rnnLayer.getOutputShapes(new Shape[]{inputShapes[0]})[0];\n", " dense.initialize(manager, dataType, new Shape(vocabSize, shape.get(shape.dimension() - 1)));\n", " }\n", "\n", " /* 我们不会实现它，因为我们不会使用它，但它是抽象块的一部分 */\n", " @Override\n", " public Shape[] getOutputShapes(Shape[] inputShapes) {\n", " return new Shape[0];\n", " }\n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 训练与预测\n", "\n", "在训练模型之前，让我们使用具有随机权重的模型进行预测。\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "Device device = manager.getDevice();\n", "RNNModel net = new RNNModel(rnnLayer, vocab.length());\n", "net.initialize(manager, DataType.FLOAT32, X.getShape());\n", "predictCh8(\"time traveller\", 10, net, vocab, device, manager);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "很明显，这种模式根本不起作用。\n", "接下来，我们使用在:numref:`sec_rnn_scratch` 中定义的相同超参数调用 `trainCh8`，并使用高级API训练我们的模型。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "int numEpochs = Integer.getInteger(\"MAX_EPOCH\", 500);\n", "\n", "int lr = 1;\n", "trainCh8((Object) net, dataset, vocab, lr, numEpochs, device, false, manager);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "与上一节相比，由于深度学习框架的高级API对代码进行了更多的优化，\n", "该模型在较短的时间内达到了较低的困惑度。\n", "\n", "\n", "## 总结\n", "\n", "* 深度学习框架的高级API提供了RNN层的实现。\n", "* 高级API的RNN层返回输出和更新的隐藏状态，其中输出不涉及输出层计算。\n", "* 与从头开始使用其实现相比，使用高级API可以更快地进行RNN训练。\n", "\n", "## 练习\n", "\n", "1. 尝试使用高级API，你能使循环神经网络模型过拟合吗？\n", "2. 如果在RNN模型中增加隐藏层的数量，会发生什么情况？你能使模型工作吗？\n", "3. 使用RNN实现 :numref:`sec_sequence` 的自回归模型。\n" ] } ], "metadata": { "kernelspec": { "display_name": "Java", "language": "java", "name": "java" }, "language_info": { "codemirror_mode": "java", "file_extension": ".jshell", "mimetype": "text/x-java-source", "name": "Java", "pygments_lexer": "java", "version": "14.0.2+12" } }, "nbformat": 4, "nbformat_minor": 4 }