代码拉取完成，页面将自动刷新
Lingvo is a framework for building neural networks in Tensorflow, particularly sequence models.
A list of publications using Lingvo can be found here.
| PyPI Version | Commit |
|---|---|
| 0.12.4 | -- |
| 0.11.0 | 6fae10077756f54beacd5c454959f20b33fd65e2 |
| 0.10.0 | 075fd1d88fa6f92681f58a2383264337d0e737ee |
| 0.9.1 | c1124c5aa7af13d2dd2b6d43293c8ca6d022b008 |
| 0.9.0 | f826e99803d1b51dccbbbed1ef857ba48a2bbefe |
| PyPI Version | Commit |
|---|---|
| 0.8.2 | 93e123c6788e934e6b7b1fd85770371becf1e92e |
| 0.7.2 | b05642fe386ee79e0d88aa083565c9a93428519e |
Details for older releases are unavailable.
NOTE: this is not a comprehensive list. Lingvo releases do not offer any guarantees regarding backwards compatibility.
Nothing here.
beam_done.done_hyps.
abc.ABCMeta should now extend
base_layer.ABCLayerMeta instead.__init__ now raises an
error.base_layer.initializer has been removed. Subclasses no longer need to
decorate their __init__ function.__init__ or
_CreateLayerVariables now raises an error.__init__. Refactor by moving the variable creation and access to
_CreateLayerVariables. The variable scope is set automatically
according to the layer name in _CreateLayerVariables.Details for older releases are unavailable.
There are two ways to set up Lingvo: installing a fixed version through pip, or cloning the repository and building it with bazel. Docker configurations are provided for each case.
If you would just like to use the framework as-is, it is easiest to just install it through pip. This makes it possible to develop and train custom models using a frozen version of the Lingvo framework. However, it is difficult to modify the framework code or implement new custom ops.
If you would like to develop the framework further and potentially contribute pull requests, you should avoid using pip and clone the repository instead.
pip:
The Lingvo pip package can be installed with
pip3 install lingvo.
See the codelab for how to get started with the pip package.
From sources:
The prerequisites are:
C++ compiler (only g++ 7.3 is officially supported), andRefer to docker/dev.Dockerfile for a set of working requirements.
git clone the repository, then use bazel to build and run targets directly.
The python -m module commands in the codelab need to be mapped onto bazel run commands.
docker:
Docker configurations are available for both situations. Instructions can be found in the comments on the top of each file.
pip:
mkdir -p /tmp/mnist
python3 -m lingvo.tools.keras2ckpt --dataset=mnist
bazel:
mkdir -p /tmp/mnist
bazel run -c opt //lingvo/tools:keras2ckpt -- --dataset=mnist
The following files will be created in /tmp/mnist:
mnist.data-00000-of-00001: 53MB.mnist.index: 241 bytes.pip:
cd /tmp/mnist
curl -O https://raw.githubusercontent.com/tensorflow/lingvo/master/lingvo/tasks/image/params/mnist.py
python3 -m lingvo.trainer --run_locally=cpu --mode=sync --model=mnist.LeNet5 --logdir=/tmp/mnist/log
bazel:
(cpu) bazel build -c opt //lingvo:trainer
(gpu) bazel build -c opt --config=cuda //lingvo:trainer
bazel-bin/lingvo/trainer --run_locally=cpu --mode=sync --model=image.mnist.LeNet5 --logdir=/tmp/mnist/log --logtostderr
After about 20 seconds, the loss should drop below 0.3 and a checkpoint will be saved, like below. Kill the trainer with Ctrl+C.
trainer.py:518] step: 205, steps/sec: 11.64 ... loss:0.25747201 ...
checkpointer.py:115] Save checkpoint
checkpointer.py:117] Save checkpoint done: /tmp/mnist/log/train/ckpt-00000205
Some artifacts will be produced in /tmp/mnist/log/control:
params.txt: hyper-parameters.model_analysis.txt: model sizes for each layer.train.pbtxt: the training tf.GraphDef.events.*: a tensorboard events file.As well as in /tmp/mnist/log/train:
checkpoint: a text file containing information about the checkpoint files.ckpt-*: the checkpoint files.Now, let's evaluate the model on the "Test" dataset. In the normal training setup the trainer and evaler should be run at the same time as two separate processes.
pip:
python3 -m lingvo.trainer --job=evaler_test --run_locally=cpu --mode=sync --model=mnist.LeNet5 --logdir=/tmp/mnist/log
bazel:
bazel-bin/lingvo/trainer --job=evaler_test --run_locally=cpu --mode=sync --model=image.mnist.LeNet5 --logdir=/tmp/mnist/log --logtostderr
Kill the job with Ctrl+C when it starts waiting for a new checkpoint.
base_runner.py:177] No new check point is found: /tmp/mnist/log/train/ckpt-00000205
The evaluation accuracy can be found slightly earlier in the logs.
base_runner.py:111] eval_test: step: 205, acc5: 0.99775392, accuracy: 0.94150388, ..., loss: 0.20770954, ...
To run a more elaborate model, you'll need a cluster with GPUs. Please refer to
third_party/py/lingvo/tasks/mt/README.md
for more information.
To train a GShard language model with one trillion parameters on GCP using
CloudTPUs v3-512 using 512-way model parallelism, please refer to
third_party/py/lingvo/tasks/lm/README.md
for more information.
To run the StarNet model using CloudTPUs on GCP, please refer to
third_party/py/lingvo/tasks/car/README.md.
Listen, Attend and Spell.
William Chan, Navdeep Jaitly, Quoc V. Le, and Oriol Vinyals. ICASSP 2016.
End-to-end Continuous Speech Recognition using Attention-based Recurrent
NN: First Results.
Jan Chorowski,
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. arXiv 2014.
DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection.
Yingwei Li, Adams Wei Yu, Tianjian Meng, Ben Caine, Jiquan Ngiam, Daiyi Peng, Junyang Shen, Bo Wu, Yifeng Lu, Denny
Zhou, Quoc V. Le, Alan Yuille, Mingxing Tan. CVPR 2022.
StarNet: Targeted Computation for Object Detection in Point Clouds.
Jiquan Ngiam, Benjamin Caine, Wei Han, Brandon Yang, Yuning Chai, Pei Sun, Yin
Zhou, Xi Yi, Ouais Alsharif, Patrick Nguyen, Zhifeng Chen, Jonathon Shlens,
and Vijay Vasudevan. arXiv 2019.
Gradient-based learning applied to document recognition.
Yann LeCun, Leon Bottou, Yoshua Bengio, and Patrick Haffner. IEEE 1998.
Exploring the Limits of Language Modeling.
Rafal Jozefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, and Yonghui
Wu. arXiv, 2016.
GShard: Scaling Giant Models with Conditional Computation and Automatic
Sharding.
Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun,
Noam Shazeer and Zhifeng Chen arXiv, 2020.
The Best of Both Worlds: Combining Recent Advances in Neural Machine
Translation.
Mia X. Chen, Orhan Firat, Ankur Bapna, Melvin Johnson, Wolfgang Macherey, George Foster, Llion
Jones, Mike Schuster, Noam Shazeer, Niki Parmar, Ashish Vaswani, Jakob
Uszkoreit, Lukasz Kaiser, Zhifeng Chen, Yonghui Wu, and Macduff Hughes.
ACL 2018.
Self-supervised and Supervised Joint Training for Resource-rich Neural
Machine Translation.
Yong Cheng, Wei Wang, Lu Jiang, and Wolfgang Macherey. ICML 2021.
Please cite this paper when referencing Lingvo.
@misc{shen2019lingvo,
title={Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling},
author={Jonathan Shen and Patrick Nguyen and Yonghui Wu and Zhifeng Chen and others},
year={2019},
eprint={1902.08295},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
此处可能存在不合适展示的内容，页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容，可点击提交进行申诉，我们将尽快为您处理。
1. 开源生态
2. 协作、人、软件
3. 评估模型