විකිපීඩියා:සැකිලි සීමා

MediaWiki has several parameters that limit the amount of data that can be included into a page. These limits concern data that is transcluded or substituted onto a page but not data directly in the source of the page itself.

The inclusion limits were put into effect on the English Wikipedia by Tim Starling on 14 August, 2006. The most important of these limits is the pre-expand include limit, which was originally 1 MB but was increased to 2 MB.

This documentation explains how and why these limits are applied, and how users can work within the limits.

Background

What is this about?

The MediaWiki software, which generates the HTML of a page from its wiki source, uses a type of parser (called a recursive descent parser) to deal with included data. As the parser does this, it keeps track of the amount of data included into the page using two counters, the pre-expand counter and the post-expand counter. When the parsing of a page begins, these counters are set to zero, but they are incremented (by a variable amount) each time data is transcluded or substituted. There are upper limits on these counters, and the parser does not allow these limits to be exceeded.

How is this done?

Whenever the parser is instructed by the source code of a page to expand a template (that is, to replace it by transclusion or substitution), it first adds together the length of the source code of the template (unexpanded) and the pre-expand counter of the page.

If this sum is more than the pre-expand limit, the template is not replaced and an error message is added as a comment in the output HTML.

Otherwise, the pre-expand counter is increased to the new value, and the source code of the template is parsed recursively in the same way. Once this is done, the parser adds together the length of the HTML generated by the template and the post-expand counter of the page.

If this sum is more than the post-expand limit, the initial template is not replaced and a different error message is added as a comment in the output HTML.

Otherwise the post-expand counter is increased to the new value, and the HTML generated by the template is added to HTML of the page body.

Because templates are expanded recursively, if the included data of a template itself transcludes or substitutes other templates, these subtemplates are also used to increase the pre-expand and post-expand counters of the page, and are not included if the limits are exceeded. Hence it is possible for a template to be only partially expanded if one of these limits is reached while the parser handles its subtemplates.

Also note that the counters are increased each time a template is expanded and added to the page body. Hence a template that occurs more than once in the page source (or any recursively included template) can contribute many times to the pre-expand and post-expand size of the page.

Why are there limits?

Pages with many templates are slow to parse. Not only is this an inconvenience for users, but it can also be used to mount a denial of service (DoS) attack on the servers, in which a page request forces the MediaWiki software to parse an unreasonably large quantity of data. The limits help to prevent this kind of attack, and ensure that pages are rendered in a reasonable time.

Working within the limits

When a page reaches the template limits, the most common solution is to make the templates shorter, using methods described below. If this isn't possible, it may be necessary to include more data directly in the page source, rather than transcluding it from templates.

When do problems arise?

The inclusion limits are most commonly reached on pages that use the same template many times, for example using one transclusion per row of a long table. The limits are reached more quickly when that template (or a template it calls) is large, e.g. because it is complicated, because it contains documentation, or because it contains data about all table rows instead of just the one concerned. Even though the amount of data that appears in the final page may be small, the total length of the source of the template is counted once per transclusion, and so the limit may be encountered sooner than expected. Pages that only include a few dozen templates are unlikely to exceed the inclusion limits, unless these templates themselves include a lot of data

How can you find out?

Once the page body is processed, if the pre-expand include counter is greater than one-thousand bytes, then an HTML comment is added towards the end of the HTML code of the page with the final values of the two counters, called the pre-expand include size and the post-expand include size of the page. For example, the page AIDS (on May 8 2007) contains the following comment in its generated HTML source:

<!-- 
Pre-expand include size: 887843 bytes
Post-expand include size: 331055 bytes
Template argument size: 244614 bytes
Maximum: 2048000 bytes
-->

Because of the way the counters are increased, these sizes will always be less than the limits. If either of these sizes is close to the limit, then it is likely that some templates have not been expanded. Each occurrence of an unexpanded template is identified in the page body by an HTML comment containing an error message.

Example

For example, assume that template {{A}} contains {{B}}{{B}}{{B}}{{B}}{{B}} and the source of template {{B}} is 100 bytes of plain text with no inclusions. Then:

Any inclusion of template {{A}} will add 25 to the pre-expand counter of the page. If there are under 25 bytes left between the pre-expand counter and its limit when template {{A}} is encountered, this template will not be expanded.
Suppose there are 250 bytes left between the pre-expand counter and its limit when template {{A}} is encountered. Then template {{A}} will be expanded, leaving 225 bytes. The first two instances of template {{B}} will be expanded, leaving 25 bytes. The remaining three instances of template {{B}} will not be expanded, because there are not enough pre-expand bytes left.
The previous two points ignore the post-expand counter. Suppose also that there are only 150 bytes left between the post-expand counter and its limit, then after the parser has expanded two of the occurrences of template {{B}}, the result the expansion of {{A}} produces more than 200 bytes of code. Since this exceeds the remaining post-expand space, {{A}} will not be replaced.

The length of HTML comments is included in the pre-expand counter but not the post-expand counter.

Using noinclude and onlyinclude

When a template is expanded, the pre-expand counter is incremented by the total length of the template's wiki source, irrespective of whether the data is inside or outside <noinclude>, <includeonly> or <onlyinclude> sections of the source. However, code which is either inside a <noinclude> section or outside an <onlyinclude> section does not get expanded, and so only the unexpanded length of these sections contributes to the pre-expand size, and these sections do not contribute at all to the post-expand size.

For example, if template {{C}} contains <noinclude>{{D}}</noinclude>, {{D}} contains 500 bytes of data, and there are 250 bytes remaining between the the pre-expand counter and its limit, then template {{C}} will be successfully expanded and its raw length of 28 bytes will be added to the pre-expand counter.

Documentation of templates

<noinclude> sections in templates are frequently used to provide documentation on the usage of the template. Because only the raw length of such sections (the length of the unexpanded wikitext) contributes to the pre-expand size when the template is included on another page, the effective size of the template can be reduced by transcluding the documentation of the template from a /doc subpage, as described in Wikipedia:Template doc page pattern (see also m:Template:documentation (talk, backlinks, edit) and [1]).

One example of this is the {{user}} template (on May 8 2007), which contains:


[[User:{{{User|{{{1|Example}}}}}}|{{{User|{{{1|''Example''}}}}}}]] 
([[User talk:{{{User|{{{1|Example}}}}}}|talk]] <small>•</small> 
[[Special:Contributions/{{{User|{{{1|Example}}}}}}|contribs]])<noinclude>
{{/doc}}
</noinclude>

The length of the source code of this template is 228 bytes, and this length is added to the pre-expand counter each time this template is included. The /doc subpage is only transcluded on the template page itself: when the template is included on another page, the {{/doc}} template is not expanded, and so it only contributes 33 bytes to the pre-expand counter: 8 for the template tag itself, 23 for the noinclude tags, and 2 for the newlines (actually, the latter could be avoided).

Conditional inclusion

It is possible for the inclusion of a template to depend on testing a condition using a parser function such as {{#ifexpr}}. One might imagine that if the test is false then the inclusion will not count towards the pre-expand size. For example, the test

{{#ifexpr:0|{{:123}}}}

is vacuously false and so the article 123 will not be included. However, the content of article 123 does count towards the pre-expand size in this case, because the arguments of parser functions are expanded before the parser function is executed (Bug 8446).

The solution to this problem is to move the transclusion outside the parser function as follows:

{{ {{#ifexpr:0|:123|x0}} }}.

Again article 123 will not be included, but now the arguments of the parser function are not transclusions, and only the template {{x0}} contributes to the pre-expand size. Since this template is empty, it contributes nothing.

The two expressions have the same effect as regards content: the first expression expands to "" and the second does as well: "". However, the difference between the two can easily be checked using "What links here" and "edit this page". In the case of this page, it is the first conditional inclusion, not the second, which causes What links to 123 and edit this page to report that the page 123 has been transcluded here.

A template for each array element

In the case of using a "large" array (say 100 or more elements) it is better to use a separate template for each array element, for example m:Template:eln de (talk, backlinks, edit): in that case the pre-expand include size of using all array elements once depends only linearly of the array size. The alternative, putting the whole array in one template with #switch, for example m:Template:n en (talk, backlinks, edit), causes the pre-expand include size of using all array elements once to be proportial to the square of the array size; this way the limit is quickly reached.

An intermediate solution between, for example, using 1 template with 100 elements and 100 templates with 1 element, would be using 10 templates with 10 elements each, giving a corresponding intermediate pre-expand include size.

On a page of 100k one can afford that the content comes from templates which are a factor 20 larger than the result. This could be a switch with a list of 10 elements (the indexes, braces and the word "#switch:" also count), but not much more. If there are multiple layers of templates each layer counts.

Avoiding identical template calls

Identical calls of template A can sometimes be avoided by calling template B with a parameter value depending on A, where B is a template which makes multiple use of this parameter value.

Special:ExpandTemplates

Tim Starling also created a related new special page ExpandTemplates at Special:ExpandTemplates. It is used to manually expand wikitext containing template calls. The calls are recursively expanded to their plain wiki-text. Special:ExpandTemplates also recursively expands all ParserFunction calls — something that cannot be achieved with substitution (see bug 2777). In some situations, it may be necessary to completely expand a template using the ExpandTemplates page and then manually copy the resulting source code into an article or template. This is most useful when expanding the template reduces its length, such as when a call of #switch containing a list of alternatives is "expanded" to a single alternative.

References

Village pump technical (discussion now moved to Wikipedia talk:Template limits)
Tim's posting on wikipedia-l