po: remove all dependencies on nicepagetitle

[ikiwiki.git] / doc / plugins / po.mdwn

diff --git a/doc/plugins/po.mdwn b/doc/plugins/po.mdwn
index e88cc3106531fd301ed99a5f2c58bddc24b91c8e..b5c5065cd48edb3adaf9bce327ca5737519e5672 100644 (file)
--- a/doc/plugins/po.mdwn
+++ b/doc/plugins/po.mdwn
@@ -5,6 +5,8 @@ This plugin adds support for multi-lingual wikis, translated with
 gettext, using [po4a](http://po4a.alioth.debian.org/).
 
 It depends on the Perl `Locale::Po4a::Po` library (`apt-get install po4a`).
 gettext, using [po4a](http://po4a.alioth.debian.org/).
 
 It depends on the Perl `Locale::Po4a::Po` library (`apt-get install po4a`).

+As detailed bellow in the security section, `po4a` is subject to
+denial-of-service attacks before version 0.35.

 
 [[!toc levels=2]]
 
 
 [[!toc levels=2]]
 


@@ -31,6 +33,12 @@ Example: `bla/page.fr.po` is the PO "message catalog" used to
 translate `bla/page.mdwn` into French; if `usedirs` is enabled, it is
 rendered as `bla/page/index.fr.html`, else as `bla/page.fr.html`
 
 translate `bla/page.mdwn` into French; if `usedirs` is enabled, it is
 rendered as `bla/page/index.fr.html`, else as `bla/page.fr.html`
 

+(In)Compatibility
+=================
+
+This plugin does not support the `indexpages` mode. If you don't know
+what it is, you probably don't care.
+

 
 Configuration
 =============
 
 Configuration
 =============


@@ -64,39 +72,40 @@ worry about excluding them explicitly from this [[ikiwiki/PageSpec]].
 Internal links
 --------------
 
 Internal links
 --------------
 

+### Links targets
+

 The `po_link_to` option in `ikiwiki.setup` is used to decide how
 internal links should be generated, depending on web server features
 and site-specific preferences.
 
 The `po_link_to` option in `ikiwiki.setup` is used to decide how
 internal links should be generated, depending on web server features
 and site-specific preferences.
 

-### Default linking behavior
+#### Default linking behavior

 
 If `po_link_to` is unset, or set to `default`, ikiwiki's default
 linking behavior is preserved: `\[[destpage]]` links to the master
 language's page.
 
 
 If `po_link_to` is unset, or set to `default`, ikiwiki's default
 linking behavior is preserved: `\[[destpage]]` links to the master
 language's page.
 

-### Link to current language
+#### Link to current language

 
 If `po_link_to` is set to `current`, `\[[destpage]]` links to the
 `destpage`'s version written in the current page's language, if
 available, *i.e.*:
 
 
 If `po_link_to` is set to `current`, `\[[destpage]]` links to the
 `destpage`'s version written in the current page's language, if
 available, *i.e.*:
 

-- `foo/destpage/index.LL.html` if `usedirs` is enabled
-- `foo/destpage.LL.html` if `usedirs` is disabled
+* `foo/destpage/index.LL.html` if `usedirs` is enabled
+* `foo/destpage.LL.html` if `usedirs` is disabled

 
 

-### Link to negotiated language
+#### Link to negotiated language

 
 If `po_link_to` is set to `negotiated`, `\[[page]]` links to the
 negotiated preferred language, *i.e.* `foo/page/`.
 
 (In)compatibility notes:
 
 
 If `po_link_to` is set to `negotiated`, `\[[page]]` links to the
 negotiated preferred language, *i.e.* `foo/page/`.
 
 (In)compatibility notes:
 

-- if `usedirs` is disabled, it does not make sense to set `po_link_to`
+* if `usedirs` is disabled, it does not make sense to set `po_link_to`

   to `negotiated`; this option combination is neither implemented
   nor allowed.
   to `negotiated`; this option combination is neither implemented
   nor allowed.

-- if the web server does not support Content Negotiation, setting
+* if the web server does not support Content Negotiation, setting

   `po_link_to` to `negotiated` will produce a unusable website.
 
   `po_link_to` to `negotiated` will produce a unusable website.
 

-

 Server support
 ==============
 
 Server support
 ==============
 


@@ -129,6 +138,10 @@ Usage
 Templates
 ---------
 
 Templates
 ---------
 

+When `po_link_to` is not set to `negotiated`, one should replace some
+occurrences of `BASEURL` with `HOMEPAGEURL` to get correct links to
+the wiki homepage.
+

 The `ISTRANSLATION` and `ISTRANSLATABLE` variables can be used to
 display things only on translatable or translation pages.
 
 The `ISTRANSLATION` and `ISTRANSLATABLE` variables can be used to
 display things only on translatable or translation pages.
 


@@ -156,11 +169,11 @@ One typically adds the following code to `templates/page.tmpl`:
 
 The following variables are available inside the loop (for every page in):
 
 
 The following variables are available inside the loop (for every page in):
 

-- `URL` - url to the page
-- `CODE` - two-letters language code
-- `LANGUAGE` - language name (as defined in `po_slave_languages`)
-- `MASTER` - is true (1) if, and only if the page is a "master" page
-- `PERCENT` - for "slave" pages, is set to the translation completeness, in percents
+* `URL` - url to the page
+* `CODE` - two-letters language code
+* `LANGUAGE` - language name (as defined in `po_slave_languages`)
+* `MASTER` - is true (1) if, and only if the page is a "master" page
+* `PERCENT` - for "slave" pages, is set to the translation completeness, in percents

 
 ### Display the current translation status
 
 
 ### Display the current translation status
 


@@ -194,14 +207,18 @@ Also, when the plugin has just been enabled, or when a page has just
 been declared as being translatable, the needed POT and PO files are
 created, and the PO files are checked into version control.
 
 been declared as being translatable, the needed POT and PO files are
 created, and the PO files are checked into version control.
 

-Discussion pages
-----------------
+Discussion pages and other sub-pages
+------------------------------------

 
 Discussion should happen in the language in which the pages are
 written for real, *i.e.* the "master" one. If discussion pages are
 enabled, "slave" pages therefore link to the "master" page's
 discussion page.
 
 
 Discussion should happen in the language in which the pages are
 written for real, *i.e.* the "master" one. If discussion pages are
 enabled, "slave" pages therefore link to the "master" page's
 discussion page.
 

+Likewise, "slave" pages are not supposed to have sub-pages;
+[[WikiLinks|wikilink]] that appear on a "slave" page therefore link to
+the master page's sub-pages.
+

 Translating
 -----------
 
 Translating
 -----------
 


@@ -211,180 +228,85 @@ interface could also be implemented at some point).
 If [[tips/untrusted_git_push]] is setup, one can edit the PO files in one's
 preferred `$EDITOR`, without needing to be online.
 
 If [[tips/untrusted_git_push]] is setup, one can edit the PO files in one's
 preferred `$EDITOR`, without needing to be online.
 

-TODO
-====
-
-Security checks
----------------
-
-### Security history
+Markup languages support
+------------------------

 
 

-The only past security issues I could find in GNU gettext and po4a
-are:
+Markdown is well supported. Some other markup languages supported by
+ikiwiki mostly work, but some pieces of syntax are not rendered
+correctly on the slave pages:

 
 

-- [CVE-2004-0966](http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2004-0966),
-  *i.e.* [Debian bug #278283](http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=278283):
-  the autopoint and gettextize scripts in the GNU gettext package
-  1.14 and later versions, as used in Trustix Secure Linux 1.5
-  through 2.1 and other operating systems, allows local users to
-  overwrite files via a symlink attack on temporary files.
-- [CVE-2007-4462](http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2007-4462):
-  `lib/Locale/Po4a/Po.pm` in po4a before 0.32 allows local users to
-  overwrite arbitrary files via a symlink attack on the
-  gettextization.failed.po temporary file.
+* [[reStructuredText|rst]]: anonymous hyperlinks and internal
+  cross-references
+* [[wikitext]]: conversion of newlines to paragraphs
+* [[creole]]: verbatim text is wrapped, tables are broken
+* [[html]] and LaTeX: not supported yet; the dedicated po4a modules
+  could be used to support them, but they would need a security audit
+* other markup languages have not been tested.

 
 

-**FIXME**: check whether this plugin would have been a possible attack
-vector to exploit these vulnerabilities.
+Security
+========

 
 

-Depending on my mood, the lack of found security issues can either
-indicate that there are none, or reveal that no-one ever bothered to
-find (and publish) them.
+[[./security]] contains a detailed security analysis of this plugin
+and its dependencies.

 
 

-### PO file features
+When using po4a older than 0.35, it is recommended to uninstall
+`Text::WrapI18N` (Debian package `libtext-wrapi18n-perl`), in order to
+avoid a potential denial of service.

 
 

-Can any sort of directives be put in po files that will cause mischief
-(ie, include other files, run commands, crash gettext, whatever)?
+TODO
+====

 
 

-> No [documented](http://www.gnu.org/software/gettext/manual/gettext.html#PO-Files)
-> directive is supposed to do so. [[--intrigeri]]
+Better links
+------------

 
 

-### Running po4a on untrusted content
+Once the fix to
+[[bugs/pagetitle_function_does_not_respect_meta_titles]] from
+[[intrigeri]]'s `meta` branch is merged into ikiwiki upstream, the
+generated links' text will be optionally based on the page titles set
+with the [[meta|plugins/meta]] plugin, and will thus be translatable.
+It will also allow displaying the translation status in links to slave
+pages. Both were implemented, and reverted in commit **FIXME**, which
+should be reverted once [[intrigeri]]'s `meta` branch is merged.

 
 

-Are there any security issues on running po4a on untrusted content?
+Robustness tests
+----------------

 
 

-To say the least, this issue is not well covered, at least publicly:
-
-- the documentation does not talk about it;
-- grep'ing the source code for `security` or `trust` gives no answer.
-
-On the other hand, a po4a developer answered my questions in
-a convincing manner, stating that processing untrusted content was not
-an initial goal, and analysing in detail the possible issues.
-
-#### Already checked
-
-- the core (`Po.pm`, `Transtractor.pm`) should be safe
-- po4a source code was fully checked for other potential symlink
-  attacks, after discovery of one such issue
-- the only external program run by the core is `diff`, in `Po.pm` (in
-  parts of its code we don't use)
-- `Locale::gettext`: only used to display translated error messages
-- Nicolas François "hopes" `DynaLoader` is safe, and has "no reason to
-  think that `Encode` is not safe"
-- Nicolas François has "no reason to think that `Encode::Guess` is not
-  safe". The po plugin nevertheless avoids using it by defining the
-  input charset (`file_in_charset`) before asking `Transtractor` to
-  read any file. NB: this hack depends on po4a internals to stay
-  the same.
-
-#### To be checked
-
-##### Locale::Po4a modules
-
-- the modules we want to use have to be checked, as not all are safe
-  (e.g. the LaTeX module's behaviour is changed by commands included
-  in the content); they may use regexps generated from the content; we
-  currently only use the `Text` module
-- the `Text` module does not run any external program
-- check that no module is loaded by `Chooser.pm`, when we tell it to
-  load the `Text` one
-- `nsgmls` is used by `Sgml.pm`
+### Enabling/disabling the plugin

 
 

-##### Text::WrapI18N
+* enabling the plugin with `po_translatable_pages` set to blacklist: **OK**
+* enabling the plugin with `po_translatable_pages` set to whitelist: **OK**
+* enabling the plugin without `po_translatable_pages` set: **OK**
+* disabling the plugin: **OK**

 
 

-`Text::WrapI18N` can cause DoS (see the
-[Debian bug #470250](http://bugs.debian.org/470250)), but it is
-optional and we do not need the features it provides.
+### Changing the plugin config

 
 

-It is loaded if available by `Locale::Po4a::Common`; looking at the
-code, I'm not sure we can prevent this at all, but maybe some symbol
-table manipulation tricks could work; overriding
-`Locale::Po4a::Common::wrapi18n` may be easier. I'm no expert at all
-in this field. Joey? [[--intrigeri]]
+* adding existing pages to `po_translatable_pages`: **OK**
+* removing existing pages from `po_translatable_pages`: **OK**
+* adding a language to `po_slave_languages`: **OK**
+* removing a language from `po_slave_languages`: **OK**
+* changing `po_master_language`: **OK**
+* replacing `po_master_language` with a language previously part of
+  `po_slave_languages`: needs two rebuilds, but **OK** (this is quite
+  a perverse test actually)

 
 

-##### Term::ReadKey
+### Creating/deleting/renaming pages

 
 

-`Term::ReadKey` is not a hard dependency in our case, *i.e.* po4a
-works nicely without it. But the po4a Debian package recommends
-`libterm-readkey-perl`, so it will probably be installed on most
-systems using the po plugin.
+All cases of master/slave page creation/deletion/rename, both via RCS
+and via CGI, have been tested.

 
 

-If `$ENV{COLUMNS}` is not set, `Locale::Po4a::Common` uses
-`Term::ReadKey::GetTerminalSize()` to get the terminal size. How safe
-is this?
+### Misc

 
 

-Part of `Term::ReadKey` is written in C. Depending on the runtime
-platform, this function use ioctl, environment, or C library function
-calls, and may end up running the `resize` command (without
-arguments).
-
-IMHO, using Term::ReadKey has too far reaching implications for us to
-be able to guarantee anything wrt. security. Since it is anyway of no
-use in our case, I suggest we define `ENV{COLUMNS}` before loading
-`Locale::Po4a::Common`, just to be on the safe side. Joey?
-[[--intrigeri]]
-
-### msgmerge
-
-`refreshpofiles()` runs this external program. A po4a developer
-answered he does "not expect any security issues from it".
-
-### Fuzzing input
+* general test with `usedirs` disabled: **OK**
+* general test with `indexpages` enabled: **not OK**
+* general test with `po_link_to=default` with `userdirs` enabled: **OK**
+* general test with `po_link_to=default` with `userdirs` disabled: **OK**

 
 

-I was not able to find any public information about gettext or po4a
-having been tested with a fuzzing program, such as `zzuf` or `fusil`.
-Moreover, some gettext parsers seem to be quite
-[easy to crash](http://fusil.hachoir.org/trac/browser/trunk/fuzzers/fusil-gettext),
-so it might be useful to bang msgmerge/po4a's heads against such
-a program in order to easily detect some of the most obvious DoS.
-[[--intrigeri]]
+Misc. bugs
+----------

 
 

-> po4a was not fuzzy-tested, but according to one of its developers,
-> "it would be really appreciated". [[--intrigeri]]
+Documentation
+-------------

 
 

-gettext/po4a rough corners
---------------------------
-
-- fix infinite loop when synchronizing two ikiwiki (when checkouts
-  live in different directories): say bla.fr.po has been updated in
-  repo2; pulling repo2 from repo1 seems to trigger a PO update, that
-  changes bla.fr.po in repo1; then pushing repo1 to repo2 triggers
-  a PO update, that changes bla.fr.po in repo2; etc.; quickly fixed in
-  `629968fc89bced6727981c0a1138072631751fee`, by disabling references
-  in Pot files. Using `Locale::Po4a::write_if_needed` might be
-  a cleaner solution. (warning: this function runs the external
-  `diff` program, have to check security)
-- new translations created in the web interface must get proper
-  charset/encoding gettext metadata, else the next automatic PO update
-  removes any non-ascii chars; possible solution: put such metadata
-  into the Pot file, and let it propagate; should be fixed in
-  `773de05a7a1ee68d2bed173367cf5e716884945a`, time will tell.
-
-Misc. improvements
-------------------
-
-### page titles
-
-Use nice page titles from meta plugin in links, as inline already
-does. This is actually a duplicate for
-[[bugs/pagetitle_function_does_not_respect_meta_titles]], which might
-be fixed by something like [[todo/using_meta_titles_for_parentlinks]].
-
-### source files format
-
-Markdown is supported, great, but what about others? The set of file
-formats supported both in ikiwiki and po4a probably is greater than
-`{markdown}`. Warning: the po4a modules are the place where one can
-expect security issues.
-
-Translation quality assurance
------------------------------
-
-Modifying a PO file via the CGI must be forbidden if the new version
-is not a valid PO file. As a bonus, check that it provides a more
-complete translation than the existing one.
-
-A new `cansave` type of hook would be needed to implement this.
-
-Note: committing to the underlying repository is a way to bypass
-this check.
+Maybe write separate documentation depending on the people it targets:
+translators, wiki administrators, hackers. This plugin may be complex
+enough to deserve this.