代码拉取完成，页面将自动刷新
<!DOCTYPE html>
<!-- saved from url=(0034)https://microsoft.github.io/XGLUE/ -->
<html lang="en">
<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<link href="./webpage_files/layui/css/layui.css" rel="stylesheet">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta name="description" content="Information about CodeXGLUE Dataset">
<meta name="author" content="Daniel Campos">
<title>CodeXGLUE</title>
<!-- Bootstrap core CSS -->
<link href="./webpage_files/bootstrap.min.css" rel="stylesheet">
<!-- Custom styles for this template -->
<link href="./webpage_files/scrolling-nav.css" rel="stylesheet">
<link href="./webpage_files/main.css" rel="stylesheet">
<link href="./webpage_files/fa-all.css" rel="stylesheet">
<link rel="stylesheet" href="./webpage_files/academicons.min.css">
<link rel="stylesheet" href="./webpage_files/18-e17dee" type="text/css" media="all">
<script type="text/javascript">
function gen_mail_to_link(lhs,rhs,subject,hypertext) {
document.write("<a href=\"mailto");
document.write(":" + lhs + "@");
document.write(rhs + "?subject=" + subject + "\">" + hypertext + "<\/a>");
}
</script>
<script type="text/javascript">
var clicked = false;
function addDownloadLink() {
if (document.getElementById("TA").checked == true) {
//Usefulness
if (!clicked) {
document.getElementById("codexgluedata").innerHTML += "<a class=\"btn btn-primary btn-md\" style='margin-right: 3px;' href=\"https://msmarco.blob.core.windows.net/xglue/xglue_public.tar.gz\">CodeXGLUE Data</a>"
clicked = true;
}
alert("Download Links are now viewable")
} else {
alert("Please Agree to our conditions to download the dataset")
}
}
</script>
<style>
code {
font-family: Consolas,"courier new";
color: black;
background-color: #f1f1f1;
padding: 2px;
font-size: 90%;
}
</style>
</head>
<body id="page-top">
<nav class="navbar navbar-expand-lg navbar-dark fixed-top topnav" id="mainNav">
<div class="container">
<link rel="shortcut icon" type="image/x-icon" href="https://microsoft.github.io/CodeXGLUE/favicon.ico">
<a class="navbar-brand js-scroll-trigger" href="https://microsoft.github.io/CodeXGLUE/#page-top">CodeXGLUE</a>
<button class="navbar-toggler collapsed" type="button" data-toggle="collapse" data-target="#navbarResponsive" aria-controls="navbarResponsive" aria-expanded="false" aria-label="Toggle navigation">
<span class="navbar-toggler-icon"></span>
</button>
<div class="collapse navbar-collapse" id="navbarResponsive">
<ul class="navbar-nav ml-auto">
<li class="nav-item">
<a class="nav-link js-scroll-trigger active" href="https://microsoft.github.io/CodeXGLUE/#page-top">Home</a>
</li>
<li class="nav-item">
<a class="nav-link js-scroll-trigger" href="https://microsoft.github.io/CodeXGLUE/#intro">Intro</a>
</li>
<li class="nav-item">
<a class="nav-link js-scroll-trigger" href="https://microsoft.github.io/CodeXGLUE/#leaderboard">Leaderboard</a>
</li>
<li class="nav-item">
<a class="nav-link js-scroll-trigger" href="https://microsoft.github.io/CodeXGLUE/#submission">Submission</a>
</li>
</ul>
</div>
</div>
</nav>
<section id="intro">
<div class="container">
<center>
<img src="./webpage_files/MicrosoftLogo.png">
</center>
<div class="row">
<p>
CodeXGLUE stands for General Language Understanding Evaluation benchmark for CODE. It includes 14 datasets for 10 diversified programming language tasks covering code-code (clone detection, defect detection, cloze test, code completion, code refinement, and code-to-code translation), text-code (natural language code search, text-to-code generation), code-text (code summarization) and text-text (documentation translation) scenarios. We provide three baseline models to support these tasks, including BERT-style pre-trained model (i.e. <a href="https://github.com/microsoft/CodeBERT">CodeBERT</a>) which is good at understanding problems, GPT-style pre-trained model which we call CodeGPT to support completion and generation problems, and Encoder-Decoder framework that supports sequence-to-sequence generation problems.
</p>
</div>
<!-- <div class="row">
<h2>Terms and Conditions</h2>
<p>The CodeXGLUE datasets are intended for non-commercial research purposes only to promote advancement in the field of artificial intelligence and related areas, and is made available free of charge without extending any license or other intellectual property rights. The dataset is provided “as is” without warranty and usage of the data has risks since we may not own the underlying rights in the documents. We are not be liable for any damages related to use of the dataset. Feedback is voluntarily given and can be used as we see fit. Upon violation of any of these terms, your rights to use the dataset will end automatically. <br> If you have questions about use of the dataset or any research outputs in your products or services, we encourage you to undertake your own independent legal review. For other questions, please feel free to contact us.</p>
<div class="checkbox">
<label>
<input type="checkbox" name="termAgreed" id="TA" value="true">
I agree to terms and conditions. Upon accepting links to dataset will become available.
</label>
</div>
</div>
<div class="row">
<button type="button" class="btn btn-primary btn-default" onclick="return addDownloadLink()">Submit</button>
</div> -->
</div>
</section>
<section id="leaderboard" class="bg-light">
<div class="container">
<div class="row">
<div class="table-responsive">
<h2>Overall Leaderboard</h2>
<table id="LB-overall" lay-filter="test"></table>
<script src="./webpage_files/layui/layui.js"></script>
<script>
layui.use('table', function(){
var myTable = layui.table;
//第一个实例
myTable.render({
elem: '#LB-overall'
,height: 312
,url: 'webpage_files/participants/overall.json' //数据接口
,page: false//开启分页
,cols: [[{align: 'center', title: '', colspan: 4}
,{align: 'center', title: 'Code-Code', colspan: 6}
,{align: 'center', title: 'Text-Code', colspan: 2}
,{align: 'center', title: 'Code-Text', colspan: 1, colGroup:true}
,{align: 'center', title: 'Text-Text', colspan: 1, colGroup:true}
,{align: 'center', title: '', colspan: 1, colGroup:true}],
[ //表头
{field: 'Rank', title: 'Rank', align: 'right', width: 70}
,{field: 'Model', title: 'Model', width: 220}
,{field: 'Organization', title: 'Organization', width: 150}
,{field: 'Submission Date', title: 'Date', width: 120, sort: true}
,{field: 'clone detection', title: 'clone detection', width: 150, sort: true}
,{field: 'defect detection', title: 'defect detections', width: 150, sort: true}
,{field: 'cloze test', title: 'cloze test', width: 120, sort: true}
,{field: 'code completion', title: 'code completion', width: 120, sort: true}
,{field: 'code refinement', title: 'code refinement', width: 120, sort: true}
,{field: 'code translation', title: 'code translation', width: 120, sort: true}
,{field: 'natural language code search', title: 'natural language code search', width: 180, sort: true}
,{field: 'code generation', title: 'code generation', width: 150, sort: true}
,{field: 'code summarization', title: 'code summarization', width: 180, sort: true}
,{field: 'documentation translation', title: 'documentation translation', width: 180, sort: true}
,{field: 'CodeXGLUE Score', title: 'CodeXGLUE Score', width: 180, sort: true}
]]
});
});
</script>
</div>
</div>
</div>
<div class="container">
<div class="row">
<div class="table-responsive">
<h2>Clone Detection (Code-Code)</h2>
<table id="LB-CloneDetection" lay-filter="test"></table>
<script>
layui.use('table', function(){
var myTable = layui.table;
//第一个实例
myTable.render({
elem: '#LB-CloneDetection'
,height: 312
,url: 'webpage_files/participants/clone_detection.json' //数据接口
,page: false//开启分页
,cols: [[{align: 'center', title: '', colspan: 4}
,{align: 'center', title: 'BigCloneBench', colspan: 3}
,{align: 'center', title: 'POJ-104', colspan: 1, colGroup:true}],
[ //表头
{field: 'Rank', title: 'Rank', align: 'right', width: 70}
,{field: 'Model', title: 'Model', width: 220}
,{field: 'Organization', title: 'Organization', width: 150}
,{field: 'Submission Date', title: 'Date', width: 120, sort: true}
,{field: 'Precision', title: 'Precision', width: 140, sort: true}
,{field: 'Recall', title: 'Recall', width: 140, sort: true}
,{field: 'F1', title: 'F1', width: 140, sort: true}
,{field: 'MAP(%)', title: 'MAP(%)', width: 150, sort: true}
]]
});
});
</script>
</div>
</div>
</div>
<div class="container">
<div class="row">
<div class="table-responsive">
<h2>Defect Detection (Code-Code)</h2>
<table id="LB-DefectDetection" lay-filter="test"></table>
<script>
layui.use('table', function(){
var myTable = layui.table;
//第一个实例
myTable.render({
elem: '#LB-DefectDetection'
,height: 312
,url: 'webpage_files/participants/defect_detection.json' //数据接口
,page: false//开启分页
,cols: [
[ //表头
{field: 'Rank', title: 'Rank', align: 'right', width: 70}
,{field: 'Model', title: 'Model', width: 220}
,{field: 'Organization', title: 'Organization', width: 150}
,{field: 'Submission Date', title: 'Date', width: 120, sort: true}
,{field: 'Accuracy', title: 'Accuracy', width: 550, sort: true}
]]
});
});
</script>
</div>
</div>
</div>
<div class="container">
<div class="row">
<div class="table-responsive">
<h2>Cloze Test (Code-Code)</h2>
<table id="LB-ClozeTest" lay-filter="test"></table>
<script>
layui.use('table', function(){
var myTable = layui.table;
//第一个实例
myTable.render({
elem: '#LB-ClozeTest'
,height: 312
,url: 'webpage_files/participants/cloze_test.json' //数据接口
,page: false//开启分页
,cols: [[{align: 'center', title: '', colspan: 4}
,{align: 'center', title: 'ClozeTest-all', colspan: 7}
,{align: 'center', title: 'ClozeTest-maxmin', colspan: 7}],
[ //表头
{field: 'Rank', title: 'Rank', align: 'right', width: 70}
,{field: 'Model', title: 'Model', width: 220}
,{field: 'Organization', title: 'Organization', width: 150}
,{field: 'Submission Date', title: 'Date', width: 120, sort: true}
,{field: 'aAll', title: 'All', width: 70, sort: true}
,{field: 'aRuby', title: 'Ruby', width: 80, sort: true}
,{field: 'aJavaScript', title: 'JavaScript', width: 110, sort: true}
,{field: 'aGo', title: 'Go', width: 70, sort: true}
,{field: 'aPython', title: 'Python', width: 90, sort: true}
,{field: 'aJava', title: 'Java', width: 80, sort: true}
,{field: 'aPHP', title: 'PHP', width: 80, sort: true}
,{field: 'mAll', title: 'All', width: 70, sort: true}
,{field: 'mRuby', title: 'Ruby', width: 80, sort: true}
,{field: 'mJavaScript', title: 'JavaScript', width: 110, sort: true}
,{field: 'mGo', title: 'Go', width: 70, sort: true}
,{field: 'mPython', title: 'Python', width: 90, sort: true}
,{field: 'mJava', title: 'Java', width: 80, sort: true}
,{field: 'mPHP', title: 'PHP', width: 80, sort: true}
]]
});
});
</script>
</div>
</div>
</div>
<div class="container">
<div class="row">
<div class="table-responsive">
<h2>Code Completion (Code-Code)</h2>
<table id="LB-CodeCompletion" lay-filter="test"></table>
<script>
layui.use('table', function(){
var myTable = layui.table;
//第一个实例
myTable.render({
elem: '#LB-CodeCompletion'
,height: 312
,url: 'webpage_files/participants/code_completion.json' //数据接口
,page: false//开启分页
,cols: [[{align: 'center', title: '', colspan: 4}
,{align: 'center', title: 'CodeCompletion-line', colspan: 4}
,{align: 'center', title: 'CodeCompletion-token', colspan: 2}],
[ //表头
{field: 'Rank', title: 'Rank', align: 'right', width: 70}
,{field: 'Model', title: 'Model', width: 220}
,{field: 'Organization', title: 'Organization', width: 150}
,{field: 'Submission Date', title: 'Date', width: 120, sort: true}
,{field: 'py150-EM', title: 'py150-EM', width: 150, sort: true}
,{field: 'py150-ES', title: 'py150-ES', width: 150, sort: true}
,{field: 'java-EM', title: 'java-EM', width: 150, sort: true}
,{field: 'java-ES', title: 'java-ES', width: 150, sort: true}
,{field: 'py150-Acc', title: 'py150-Acc', width: 150, sort: true}
,{field: 'java-Acc', title: 'java-Acc', width: 150, sort: true}
]]
});
});
</script>
</div>
</div>
</div>
<div class="container">
<div class="row">
<div class="table-responsive">
<h2>Code Refinement (Code-Code)</h2>
<table id="LB-CodeRefinement" lay-filter="test"></table>
<script>
layui.use('table', function(){
var myTable = layui.table;
//第一个实例
myTable.render({
elem: '#LB-CodeRefinement'
,height: 312
,url: 'webpage_files/participants/code_refinement.json' //数据接口
,page: false//开启分页
,cols: [[{align: 'center', title: '', colspan: 4}
,{align: 'center', title: 'small test set', colspan: 3}
,{align: 'center', title: 'medium test set', colspan: 3}],
[ //表头
{field: 'Rank', title: 'Rank', align: 'right', width: 70}
,{field: 'Model', title: 'Model', width: 220}
,{field: 'Organization', title: 'Organization', width: 150}
,{field: 'Submission Date', title: 'Date', width: 120, sort: true}
,{field: 'sBLEU', title: 'BLEU', width: 90, sort: true}
,{field: 'sAcc(%)', title: 'Acc(%)', width: 90, sort: true}
,{field: 'sCodeBLEU', title: 'CodeBLEU', width: 120, sort: true}
,{field: 'mBLEU', title: 'BLEU', width: 90, sort: true}
,{field: 'mAcc(%)', title: 'Acc(%)', width: 90, sort: true}
,{field: 'mCodeBLEU', title: 'CodeBLEU', width: 120, sort: true}
]]
});
});
</script>
</div>
</div>
</div>
<div class="container">
<div class="row">
<div class="table-responsive">
<h2>Code Translation (Code-Code)</h2>
<table id="LB-Code2CodeTranslation" lay-filter="test"></table>
<script>
layui.use('table', function(){
var myTable = layui.table;
//第一个实例
myTable.render({
elem: '#LB-Code2CodeTranslation'
,height: 312
,url: 'webpage_files/participants/code2code_translation.json' //数据接口
,page: false//开启分页
,cols: [[{align: 'center', title: '', colspan: 4}
,{align: 'center', title: 'Java to C#', colspan: 3}
,{align: 'center', title: 'C# to Java', colspan: 3}],
[ //表头
{field: 'Rank', title: 'Rank', align: 'right', width: 70}
,{field: 'Model', title: 'Model', width: 220}
,{field: 'Organization', title: 'Organization', width: 150}
,{field: 'Submission Date', title: 'Date', width: 120, sort: true}
,{field: 'jBLEU', title: 'BLEU', width: 90, sort: true}
,{field: 'jAcc(%)', title: 'Acc(%)', width: 90, sort: true}
,{field: 'jCodeBLEU', title: 'CodeBLEU', width: 120, sort: true}
,{field: 'cBLEU', title: 'BLEU', width: 90, sort: true}
,{field: 'cAcc(%)', title: 'Acc(%)', width: 90, sort: true}
,{field: 'cCodeBLEU', title: 'CodeBLEU', width: 120, sort: true}
]]
});
});
</script>
</div>
</div>
</div>
<div class="container">
<div class="row">
<div class="table-responsive">
<h2>Natural Language Code Search (Text-Code)</h2>
<table id="LB-NL-code-search" lay-filter="test"></table>
<script>
layui.use('table', function(){
var myTable = layui.table;
//第一个实例
myTable.render({
elem: '#LB-NL-code-search'
,height: 312
,url: 'webpage_files/participants/code_search.json' //数据接口
,page: false//开启分页
,cols: [[{align: 'center', title: '', colspan: 4}
,{align: 'center', title: 'NL-code-search-Adv', colspan: 1, colGroup:true}
,{align: 'center', title: 'NL-code-search-WebQuery', colspan: 2}],
[ //表头
{field: 'Rank', title: 'Rank', align: 'right', width: 70}
,{field: 'Model', title: 'Model', width: 220}
,{field: 'Organization', title: 'Organization', width: 150}
,{field: 'Submission Date', title: 'Date', width: 120, sort: true}
,{field: 'MRR', title: 'MRR', sort: true}
,{field: 'Acc-WebQuery', title: 'Accuracy', sort: true}
,{field: 'F1-WebQuery', title: 'F1', sort: true}
]]
});
});
</script>
</div>
</div>
</div>
<div class="container">
<div class="row">
<div class="table-responsive">
<h2>Code Generation (Text-Code)</h2>
<table id="LB-Text2CodeGeneration" lay-filter="test"></table>
<script>
layui.use('table', function(){
var myTable = layui.table;
//第一个实例
myTable.render({
elem: '#LB-Text2CodeGeneration'
,height: 312
,url: 'webpage_files/participants/text2code_generation.json' //数据接口
,page: false//开启分页
,cols: [[{align: 'center', title: '', colspan: 4}
,{align: 'center', title: 'Text2Code Generation', colspan: 3}],
[ //表头
{field: 'Rank', title: 'Rank', align: 'right', width: 70}
,{field: 'Model', title: 'Model', width: 220}
,{field: 'Organization', title: 'Organization', width: 150}
,{field: 'Submission Date', title: 'Date', width: 120, sort: true}
,{field: 'EM', title: 'EM', sort: true}
,{field: 'BLEU', title: 'BLEU', sort: true}
,{field: 'CodeBLEU', title: 'CodeBLEU', sort: true}
]]
,done: function (res, curr, count) {
$('.layui-table').css("width","100%");
$("th[data-field='bar']").css("border-right",'none');
}
});
});
</script>
</div>
</div>
</div>
<div class="container">
<div class="row">
<div class="table-responsive">
<h2>Code Summarization (Code-Text)</h2>
<table id="LB-Code2TextGeneration" lay-filter="test"></table>
<script>
layui.use('table', function(){
var myTable = layui.table;
//第一个实例
myTable.render({
elem: '#LB-Code2TextGeneration'
,height: 312
,url: 'webpage_files/participants/code2text_generation.json' //数据接口
,page: false//开启分页
,cols: [[ //表头
{field: 'Rank', title: 'Rank', align: 'right', width: 70}
,{field: 'Model', title: 'Model', width: 220}
,{field: 'Organization', title: 'Organization', width: 150}
,{field: 'Submission Date', title: 'Date', width: 120, sort: true}
,{field: 'All', title: 'All', width: 70, sort: true}
,{field: 'Ruby', title: 'Ruby', width: 80, sort: true}
,{field: 'JavaScript', title: 'JavaScript', width: 110, sort: true}
,{field: 'Go', title: 'Go', width: 70, sort: true}
,{field: 'Python', title: 'Python', width: 90, sort: true}
,{field: 'Java', title: 'Java', width: 90, sort: true}
,{field: 'PHP', title: 'PHP', width: 80, sort: true}
]]
});
});
</script>
</div>
</div>
</div>
<div class="container">
<div class="row">
<div class="table-responsive">
<h2>Documentation Translation (Text-Text)</h2>
<table id="LB-Text2TextGeneration" lay-filter="test"></table>
<script>
layui.use('table', function(){
var myTable = layui.table;
//第一个实例
myTable.render({
elem: '#LB-Text2TextGeneration'
,height: 312
,url: 'webpage_files/participants/text2text_generation.json' //数据接口
,page: false//开启分页
,cols: [[{align: 'center', title: '', colspan: 4}
,{align: 'center', title: 'Translation Direction', colspan: 8}],
[ //表头
{field: 'Rank', title: 'Rank', align: 'right', width: 70}
,{field: 'Model', title: 'Model', width: 220}
,{field: 'Organization', title: 'Organization', width: 150}
,{field: 'Submission Date', title: 'Date', width: 120, sort: true}
,{field: 'EN->DA', title: 'EN->DA', width: 100, sort: true}
,{field: 'EN->LV', title: 'EN->LV', width: 100, sort: true}
,{field: 'EN->NO', title: 'EN->NO', width: 100, sort: true}
,{field: 'EN->ZH', title: 'EN->ZH', width: 100, sort: true}
,{field: 'DA->EN', title: 'DA->EN', width: 100, sort: true}
,{field: 'LV->EN', title: 'LV->EN', width: 100, sort: true}
,{field: 'NO->EN', title: 'NO->EN', width: 100, sort: true}
,{field: 'ZH->EN', title: 'ZH->EN', width: 100, sort: true}
]]
});
});
</script>
</div>
</div>
</div>
</section>
<section id="submission" class="bg-light">
<div class="container">
<div class="row">
<div class="col-md-12 mx-auto">
<h2>CodeXGLUE Submission Instructions</h2>
<p>Once you have built a model that meets your expectations on evaluation with the dev set, you can submit your test results to get official evaluation on the test set. To ensure the integrity of the official test results, we do not release the correct answers for test set to the public. To submit your model for official evaluation on the test set, follow the below steps:</p>
<ol style="margin-left: 30px;">
<li>1. Generate your prediction output for the dev set.</li>
<li>2. Run the official evaluation methodologies found in the task specific git repo and verify your systems are running as expected.</li>
<li>3. Generate your prediction output for the test set.</li>
<li>4. Submit the following information by emailing to <b>codexglue@microsoft.com</b> </li>
</ol>
<p>Your email should include:</p>
<ol style="margin-left: 30px;">
<li>1. Prediction results on test set. <b>[Required]</b></li>
<li>2. Prediction results on dev set. <b>[Recommended]</b></li>
<li>3. Individual/Team Name: Name of the individual or the team to appear in the leaderboard. <b>[Required]</b></li>
<li>4. Individual/Team Institution: Name of the institution of the individual or the team to appear in the leaderboard. <b>[Optional]</b></li>
<li>5. Model code: Training code for the model. <b>[Recommended]</b></li>
<li>6. Model information: Name of the model/technique to appear in the leaderboard. <b>[Required]</b></li>
<li>7. Paper Information: Name, Citation, URL of the paper if model is from a published work to appear in the leaderboard. <b>[Optional]</b></li>
</ol>
<p>To avoid "P-hacking" we discourage too many submissions from the same group in a short period of time.</p>
</div>
</div>
</div>
</section>
<section id="about" class="bg-light">
<div class="container">
<div class="row">
<div class="col-md-12 mx-auto">
<h2>Relevant Links</h2>
<a class="btn btn-primary btn-md" style="margin-right: 3px;" href="https://github.com/microsoft/CodeXGLUE">CodeXGLUE Github</a>
<a class="btn btn-primary btn-md" style="margin-right: 3px;" href="5https://arxiv.org/abs/???">CodeXGLUE Paper</a>
<a class="btn btn-primary btn-md" style="margin-right: 3px;" href="https://arxiv.org/abs/2002.08155">CodeBERT Paper</a>
<a class="btn btn-primary btn-md" style="margin-right: 3px;" href="https://github.com/microsoft/CodeBERT">CodeBERT Github</a>
<div class="row"> </div>
<div id="codexgluedata"> </div>
</div>
</div>
</div>
</section>
<section id="about" class="bg-light">
<div class="container">
<div class="row">
<div class="col-md-12 mx-auto">
<h2>How to cite</h2>
<p></p>
<p>To cite CodeXGLUE: </p>
<div style="overflow-x:scroll; white-space: nowrap">
<code>@article{CodeXGLUE,<br>
title={CodeXGLUE: An Open Challenge for Code Intelligence},<br>
journal={arXiv},<br>
year={2020},<br>
}</code><br>
</div>
<p></p>
<p>If you only use part of the CodeXGLUE dataset, please also cite each seperated dataset.</p>
<!-- <div style="overflow-x:scroll; white-space: nowrap"> -->
<!-- <code>@article{husain2019codesearchnet,<br>
title={CodeSearchNet Challenge: Evaluating the State of Semantic Code Search},<br>
author={Husain, Hamel and Wu, Ho-Hsiang and Gazit, Tiferet and Allamanis, Miltiadis and Brockschmidt, Marc},<br>
journal={arXiv preprint arXiv:1909.09436},<br>
year={2019}<br>
}</code> <br><br>
<code>@article{DBLP:journals/corr/abs-1812-08693,<br>
title= {An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation},<br>
author= {Michele Tufano, Cody Watson, Gabriele Bavota, Massimiliano Di Penta, Martin White and Denys Poshyvanyk},<br>
journal= {2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE)},<br>
year= {2018}<br>
} </code> <br><br>
<code>@article{raychev2016probabilistic, <br>
title={Probabilistic Model for Code with Decision Trees},<br>
author={Raychev, Veselin and Bielik, Pavol and Vechev, Martin},<br>
journal={ACM SIGPLAN Notices},<br>
pages={731--747},<br>
year={2016},<br>
publisher={ACM New York, NY, USA}<br>
} </code> <br><br>
<code>@inproceedings{allamanis2013mining, <br>
title={Mining Source Code Repositories at Massive Scale using Language Modeling},<br>
author={Allamanis, Miltiadis and Sutton, Charles},<br>
booktitle={2013 10th Working Conference on Mining Software Repositories (MSR)},<br>
pages={207--216},<br>
year={2013},<br>
organization={IEEE}<br>
} </code> <br><br>
<code>@inproceedings{Zhou2019DevignEV,<br>
title={Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks},<br>
author={YaQin Zhou and Shangqing Liu and Jingkai Siow and Xiaoning Du and Yang Liu},<br>
booktitle={NeurIPS},<br>
year={2019}<br>
}</code> <br><br>
<code>@inproceedings{Mou2016ConvolutionalNN,<br>
title={Convolutional Neural Networks over Tree Structures for Programming Language Processing},<br>
author={Lili Mou and G. Li and L. Zhang and T. Wang and Zhi Jin},<br>
booktitle={AAAI},<br>
year={2016}<br>
}</code> <br><br>
<code>@article{Wang2020DetectingCC,<br>
title={Detecting Code Clones with Graph Neural Network and Flow-Augmented Abstract Syntax Tree},<br>
author={Wenhan Wang and Ge Li and Bo Ma and Xin Xia and Zhi Jin},<br>
journal={2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER)},<br>
year={2020},<br>
pages={261-271}<br>
}</code> <br><br>
<code>@article{iyer2018mapping,<br>
title={Mapping Language to Code in Programmatic Context},<br>
author={Iyer, Srinivasan and Konstas, Ioannis and Cheung, Alvin and Zettlemoyer, Luke},<br>
journal={Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},<br>
pages={1643–1652},<br>
year={2018}<br>
}</code> <br><br>
<code>@inproceedings{yao2018staqc,<br>
title={StaQC: A Systematically Mined Question-Code Dataset from Stack Overflow},<br>
author={Yao, Ziyu and Weld, Daniel S and Chen, Wei-Peng and Sun, Huan},<br>
booktitle={Proceedings of the 2018 World Wide Web Conference},<br>
pages={1693--1703},<br>
year={2018}<br>
} </code> <br><br>
<code>@article{feng2020codebert,<br>
title={CodeBERT: A Pre-Trained Model for Programming and Natural Languages},<br>
author={Feng, Zhangyin and Guo, Daya and Tang, Duyu and Duan, Nan and Feng, Xiaocheng and Gong, Ming and Shou, Linjun and Qin, Bing and Liu, Ting and Jiang, Daxin and others},<br>
journal={arXiv preprint arXiv:2002.08155},<br>
year={2020}<br>
}</code> -->
<!-- </div> -->
<h2> </h2>
</div>
</div>
</div>
</section>
<!-- Bootstrap core JavaScript -->
<script src="./webpage_files/jquery.min.js.download"></script>
<script src="./webpage_files/bootstrap.bundle.min.js.download"></script>
<!-- Plugin JavaScript -->
<script src="./webpage_files/jquery.easing.min.js.download"></script>
<script src="./webpage_files/scrolling-nav.js.download"></script>
<script>
var rankings = ['codexgluetable']
for(var j=0; j < rankings.length; j++) {
var rows = document.getElementById(rankings[j]).rows;
for(var i = 1, td; i < rows.length; i++){
td = document.createElement('td');
td.appendChild(document.createTextNode(i));
rows[i].insertBefore(td, rows[i].firstChild);
}
}
</script>
</body></html>
此处可能存在不合适展示的内容，页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容，可点击提交进行申诉，我们将尽快为您处理。