web commit by tuomov: A note on getctime optimisation

[ikiwiki.git] / doc / patchqueue / index.html_allowed.mdwn

Instead of having files foo.html "in front of" foo/, I prefer to have 
foo/index.html.

I independently implemented a similar, but smaller patch.
(It's smaller because I only care about rendering; not CGI, for example.)
The key to this patch is that "A/B/C" is treated as equivalent
to "A/B/C/index".
Here it is:  --Per Bothner

    --- IkiWiki/Render.pm~  2007-01-11 15:01:51.000000000 -0800
    +++ IkiWiki/Render.pm   2007-02-02 22:24:12.000000000 -0800
    @@ -60,9 +60,9 @@
            foreach my $dir (reverse split("/", $page)) {
                    if (! $skip) {
                            $path.="../";
    -                       unshift @ret, { url => $path.htmlpage($dir), page => pagetitle($dir) };
    +                       unshift @ret, { url => abs2rel(htmlpage(bestlink($page, $dir)), dirname($page)), page => pagetitle($dir) };
                    }
    -               else {
    +               elsif ($dir ne "index") {
                            $skip=0;
                    }
            }

    --- IkiWiki.pm~ 2007-01-12 12:47:09.000000000 -0800
    +++ IkiWiki.pm  2007-02-02 18:02:16.000000000 -0800
    @@ -315,6 +315,12 @@
                    elsif (exists $pagecase{lc $l}) {
                            return $pagecase{lc $l};
                     }
    +               else {
    +                   my $lindex = $l . "/index";
    +                   if (exists $links{$lindex}) {
    +                       return $lindex;
    +               }
    +               }
             } while $cwd=~s!/?[^/]+$!!;
     
            if (length $config{userdir} && exists $links{"$config{userdir}/".lc($link)}) {

Note I handle setting the url; slightly differently.
Also note that an initial "index" is ignored.  I.e. a
page "A/B/index.html" is treated as "A/B".

> Actually, your patch is shorter because it's more elegant and better :)
> I'm withdrawing my old patch, because yours is much more in line with
> ikiwiki's design and architecture.
> I would like to make one suggestion to your patch, which is:

    diff -urX ignorepats clean-ikidev/IkiWiki/Plugin/inline.pm ikidev/IkiWiki/Plugin/inline.pm
    --- clean-ikidev/IkiWiki/Plugin/inline.pm   2007-02-25 12:26:54.099113000 -0800
    +++ ikidev/IkiWiki/Plugin/inline.pm 2007-02-25 14:55:21.163340000 -0800
    @@ -154,7 +154,7 @@
                        $link=htmlpage($link) if defined $type;
                        $link=abs2rel($link, dirname($params{destpage}));
                        $template->param(pageurl => $link);
    -                   $template->param(title => pagetitle(basename($page)));
    +                   $template->param(title => titlename($page));
                        $template->param(ctime => displaytime($pagectime{$page}));

                        if ($actions) {
    @@ -318,7 +318,7 @@
                my $pcontent = absolute_urls(get_inline_content($p, $page), $url);

                $itemtemplate->param(
    -                   title => pagetitle(basename($p), 1),
    +                   title => titlename($p, 1),
                        url => $u,
                        permalink => $u,
                        date_822 => date_822($pagectime{$p}),
    diff -urX ignorepats clean-ikidev/IkiWiki/Render.pm ikidev/IkiWiki/Render.pm
    --- clean-ikidev/IkiWiki/Render.pm  2007-02-25 12:26:54.745833000 -0800
    +++ ikidev/IkiWiki/Render.pm        2007-02-25 14:54:01.564715000 -0800
    @@ -110,7 +110,7 @@
        $template->param(
                title => $page eq 'index'
                        ? $config{wikiname}
    -                   : pagetitle(basename($page)),
    +                   : titlename($page),
                wikiname => $config{wikiname},
                parentlinks => [parentlinks($page)],
                content => $content,
    diff -urX ignorepats clean-ikidev/IkiWiki.pm ikidev/IkiWiki.pm
    --- clean-ikidev/IkiWiki.pm 2007-02-25 12:26:58.812850000 -0800
    +++ ikidev/IkiWiki.pm       2007-02-25 15:05:22.328852000 -0800
    @@ -192,6 +192,12 @@
        return $untainted;
     } #}}}

    +sub titlename($;@) { #{{{
    +   my $page = shift;
    +   $page =~ s!/index$!!;
    +   return pagetitle(basename($page), @_);
    +} #}}}
    +
     sub basename ($) { #{{{
        my $file=shift;


> This way foo/index gets "foo" as its title, not "index". --Ethan

---

How about doing the index stuff only on the output side? (Or does the latter patch do it? I haven't tried them.) That is, render every `foo.type` for the rendered types (mdwn etc.) as `foo/index.html`, generating links to `foo/` instead of `foo.html`, but not earlier than the point where the .html as presently appended to the page name. Then you just flip a build time option on an existing wiki without any changes to that, and the pages appear elsewhere. The `index.type` files might be left out of this scheme, though (and the top-level one, of course, has to). --[[tuomov]]

> Well, get around to wasting time on it after all, and [here's the patch](http://iki.fi/tuomov/use_dirs.diff). The `-use_dirs` option will cause everything to be rendered inside directories. There may still be some problems with it, that need looking into (it doesn't e.g. check for conflicts between foo/index.mdwn and foo.mdwn), but seems to work well enough for me... The patch also improves, I think, the parentlinks code a little, as it uses generic routines to actually find the target location now. The only places where the `use_dirs` option is used is `htmlpage`, in fact, although other specific kludges needed to be removed from other points in the code.

>> FWIW, [use_dirs.diff](http://iki.fi/tuomov/use_dirs.diff) applies cleanly, and works well for me. Given that it makes this behaviour optional, how about merging it? I have some follow-up patches which I'm sitting on for now. ;-) -- Ben

>>> How do you apply a patch created by svn diff? I've been curious about this for a long time. The use_dirs patch looks OK but I'd like to play with it. --Ethan

>>>> Just do `svn co svn://ikiwiki.kitenet.net/ikiwiki/trunk ikiwiki` then `cd ikiwiki && patch -p0 <use_dirs.diff`. :-) Same would work with a tarball as well.   

>>>>> Sorry, I'm dumb. I'm so used to doing -p1 that doing -p0 never occurred to me; I thought the patch format generated by svn diff was just "wrong". --Ethan

----

First pass over Tumov's patch -- which doesn't cleanly apply anymore, so
I'll attach an updated and modified version below. --[[Joey]]

* As we discussed in email, this will break handling of `foo/index.mdwn`
  pages. Needs to be changed to generate `foo/index/index.html` for such
  pages (though not for the toplevel `index`).

  >> Can someone elaborate on this? What's broken about it? Will pages
  >> foo/index/index.html include foo/index in their parentlinks? --Ethan

  >>> Presently the patch does not move `foo/index.type` as `foo/index/index.html`, but renders
  >>> it as `foo/index.html`, not because I particularly want that (except for the top-level one, of
  >>> course), but because it could be done :). This, however, conflicts with a `foo.mdwn`
  >>> rendered as `foo/index.html`. The easiest and cleanest way to fix this, is to simply
  >>> not handle `index` in such a special manner -- except for the top-level one. --[[tuomov]]

  >>>> Oh, I see, this patch doesn't address wanting to use foo/index.mdwn as 
  >>>> an input page. Hmm. --Ethan

  >>>>> No, it doesn't. I originally also was after that, but after discussing the
  >>>>> complexities of supporting that with Joey, came up with this simpler scheme
  >>>>> without many of those issues. It is the output that I primarily care about, anyway,
  >>>>> and I do, in fact, find the present input file organisation quite nice. The output
  >>>>> locations just aren't very good for conversion of an existing site to ikiwiki, and do
  >>>>> make for rather ugly URLs with the .html extensions. (I do often type some URLs
  >>>>> out of memory, when they're gone from the browser's completion history, and the
  >>>>> .html makes that more laboursome.)

  >>>>>> I support your decision, but now this wiki page serves two different patches :).
  >>>>>> Can we split them somehow?
  >>>>>> What are the complexities involved?
  >>>>>> I think I overcomplicated it a little with my patch, and Per Bothner's gets 
  >>>>>> much closer to the heart of it. --Ethan

* This does make the resulting wikis much less browsable directly on the
  filesystem, since `dir` to `dir/index.html` conversion is only handled by web
  servers and so you end up browsing to a directory index all the time.
  Wouldn't it be better to make the links themselves include the index.html?
  (Although that would mean that [[bugs/broken_parentlinks]] would not be
  fixed en passant by this patch..)

     > Yes, the sites are not that browsable on the FS (I blame the browsers
     > for being stupid!), but linking to the directory produces so much
     > cleaner URLs for the Web, that I specifically want it. This is,
     > after all, an optional arrangement. 

     >> It's optional for *now* ... I suppose that I could make adding the
     >> index.html yet another option. I'm not _that_ fond of optioons
     >> however. --[[Joey]]

     >>> It is worth noting, that with this patch, you _can_ render the local
     >>> copy in the present manner, while rendering the Web copy under
     >>> directories. So no extra options are really needed for local browsing, 
     >>> unless you also want to serve the same copy over the Web, which I
     >>> doubt. --[[tuomov]]

* I suggest keeping the links using foo/index.html in the html file, but use
  JavaScript to fix the links onload time - but only if the protocol is http or https.
  This provides nice links without the "index.html" when served by a
  web server, but degrades nicely when using a file: url, or when JavaScript
  is disabled.  --Per Bothner

         function fixLinks ( ) {
           var scheme = location.protocol;
           if (scheme!="http:" && scheme!="https:") return;
           var links = document.getElementsByTagName("a");
           for (var i = links.length; --i >= 0; )
             links[i].href = links[i].href.replace(/[/]index.html/,"");
         }

* Some of the generated links are missing the trailing / , which is
  innefficient since it leads to a http redirect when clicking on that
  link. Seems to be limited to ".." links, and possibly only to
  parentlinks. (Already fixed it for "." links.)

      > The solution seems to be to add to `urlto` the following snippet,
      > which might also help with the next point. (Sorry, no updated patch
      > yet. Should be on my way out in the cold anyway...)

        if ( !length $to ) {
                return baseurl($from);
        }
 
      >> Indeed, this brings the number of abs2rels closer to par, as well
      >> as fixing the .. links. --[[Joey]]

* It calles abs2rel about 16% more often with the patch, which makes it
  a bit slower, since abs2rel is not very efficient. (This omits abs2rel
  calls that might be memoized away already.) This seems to be due to one
  extra abs2rel for the toplevel wiki page due to the nicely cleaned up code
  in `parentlinks` -- so I'm not really complaining.. Especially since the
  patch adds a new nice memoizable `urlto`.
* The rss page name generation code seems unnecesarily roundabout, I'm sure
  that can be cleaned up somehow, perhaps by making `htmlpage` more
  generic.

     > Something like `targetpage(basename, extension)`?

     >> Yes exactly. It might also be possible to remove htmlpage from the
     >> plugin interface entirely (in favour of urlto), which would be a
     >> good time to make such a changes. Not required to accept this patch
     >> though.

     >>> [...] in fact, all uses of htmlpage in the plugins are used to
     >>> construct an absolute address: the absolute url in most cases, so an `absurl`
     >>> call could be added to be used instead of htmlpage
     >>> --[[tuomov]]

     >>>> Or it could use urlto("index", $page) instead. --[[Joey]]

     >>>>> That is, however, a relative URL, and maybe an absolute one
     >>>>> is wanted. Perhaps `urlto($targetpage)` should return the
     >>>>> absolute version --[[tuomov]]

* > and something else in the
  > aggregate plugin (above), that I also think isn't what's wanted:
  > aren't `foo.html` pages also "rendered", so that they get moved as `foo/index.html`?
  > --[[tuomov]]

  >> Yes, the aggregate plugin will save the files as foo.html in the
  >> sourcedir, and that will result in foo/index.html in the web site, same
  >> as any other page. --[[Joey]]

* `img.pm` makes some assumptions about name of the page that will be
  linking to the image, which are probably broken.

* The changes to htmlpage's behavior probably call for the plugin
  interface version number to be changed.

Latest version of my patch... with most of the stuff that's been discussed, including `targetpage`.
Also available [here](http://iki.fi/tuomov/use_dirs-20070221.diff). (BTW, this posting, applying, and
updating of plain-old-diffs containing all the previous changes is starting to be painful. Reminds
me why I use darcs..) --[[tuomov]]

<pre>
Index: IkiWiki.pm
===================================================================
--- IkiWiki.pm  (revision 2806)
+++ IkiWiki.pm  (working copy)
@@ -14,7 +14,7 @@
 use Exporter q{import};
 our @EXPORT = qw(hook debug error template htmlpage add_depends pagespec_match
                  bestlink htmllink readfile writefile pagetype srcfile pagename
-                 displaytime will_render gettext
+                 displaytime will_render gettext urlto targetpage
                  %config %links %renderedfiles %pagesources);
 our $VERSION = 1.02; # plugin interface version, next is ikiwiki version
 our $version='unknown'; # VERSION_AUTOREPLACE done by Makefile, DNE
@@ -73,6 +73,7 @@
        sslcookie => 0,
        httpauth => 0,
        userdir => "",
+       usedirs => 0
 } #}}}
    
 sub checkconfig () { #{{{
@@ -224,10 +225,21 @@
        return $page;
 } #}}}
 
+sub targetpage ($$) { #{{{
+       my $page=shift;
+       my $ext=shift;
+       
+       if (! $config{usedirs} || $page =~ /^index$/ ) {
+               return $page.".".$ext;
+       } else {
+               return $page."/index.".$ext;
+       }
+} #}}}
+
 sub htmlpage ($) { #{{{
        my $page=shift;
-
-       return $page.".html";
+       
+       return targetpage($page, "html");
 } #}}}
 
 sub srcfile ($) { #{{{
@@ -393,6 +405,7 @@
 
        return "$config{url}/" if ! defined $page;
        
+       $page=htmlpage($page);
        $page=~s/[^\/]+$//;
        $page=~s/[^\/]+\//..\//g;
        return $page;
@@ -422,6 +435,32 @@
                        $config{timeformat}, localtime($time)));
 } #}}}
 
+sub beautify_url ($) { #{{{
+       my $url=shift;
+
+       $url =~ s!/index.html$!/!;
+       $url =~ s!^$!./!; # Browsers don't like empty links...
+
+       return $url;
+} #}}}
+
+sub urlto ($$) { #{{{
+       my $to=shift;
+       my $from=shift;
+
+       if (! length $to) {
+               return beautify_url(baseurl($from));
+       }
+
+       if (! grep { $_ eq $to } map { @{$_} } values %renderedfiles) {
+               $to=htmlpage($to);
+       }
+
+       my $link = abs2rel($to, dirname(htmlpage($from)));
+
+       return beautify_url($link);
+} #}}}
+
 sub htmllink ($$$;@) { #{{{
        my $lpage=shift; # the page doing the linking
        my $page=shift; # the page that will contain the link (different for inline)
@@ -457,7 +496,8 @@
                        "\">?</a>$linktext</span>"
        }
        
-       $bestlink=abs2rel($bestlink, dirname($page));
+       $bestlink=abs2rel($bestlink, dirname(htmlpage($page)));
+       $bestlink=beautify_url($bestlink);
        
        if (! $opts{noimageinline} && isinlinableimage($bestlink)) {
                return "<img src=\"$bestlink\" alt=\"$linktext\" />";
Index: IkiWiki/Render.pm
===================================================================
--- IkiWiki/Render.pm   (revision 2806)
+++ IkiWiki/Render.pm   (working copy)
@@ -32,8 +32,8 @@
        my @links;
        return unless $backlinks{$page};
        foreach my $p (keys %{$backlinks{$page}}) {
-               my $href=abs2rel(htmlpage($p), dirname($page));
-                       
+               my $href=urlto($p, $page);
+                
                # Trim common dir prefixes from both pages.
                my $p_trimmed=$p;
                my $page_trimmed=$page;
@@ -55,18 +55,14 @@
        my @ret;
        my $pagelink="";
        my $path="";
-       my $skip=1;
+       my $title=$config{wikiname};
+       
        return if $page eq 'index'; # toplevel
-       foreach my $dir (reverse split("/", $page)) {
-               if (! $skip) {
-                       $path.="../";
-                       unshift @ret, { url => $path.htmlpage($dir), page => pagetitle($dir) };
-               }
-               else {
-                       $skip=0;
-               }
+       foreach my $dir (split("/", $page)) {
+               push @ret, { url => urlto($path, $page), page => $title };
+               $path.="/".$dir;
+               $title=pagetitle($dir);
        }
-       unshift @ret, { url => length $path ? $path : ".", page => $config{wikiname} };
        return @ret;
 } #}}}
 
Index: IkiWiki/Plugin/inline.pm
===================================================================
--- IkiWiki/Plugin/inline.pm    (revision 2806)
+++ IkiWiki/Plugin/inline.pm    (working copy)
@@ -110,8 +110,8 @@
 
        add_depends($params{page}, $params{pages});
 
-       my $rssurl=rsspage(basename($params{page}));
-       my $atomurl=atompage(basename($params{page}));
+       my $rssurl=basename(rsspage($params{page}));
+       my $atomurl=basename(atompage($params{page}));
        my $ret="";
 
        if (exists $params{rootpage} && $config{cgiurl}) {
@@ -150,10 +150,7 @@
                        # Don't use htmllink because this way the
                        # title is separate and can be overridden by
                        # other plugins.
-                       my $link=bestlink($params{page}, $page);
-                       $link=htmlpage($link) if defined $type;
-                       $link=abs2rel($link, dirname($params{destpage}));
-                       $template->param(pageurl => $link);
+                       $template->param(pageurl => urlto(bestlink($params{page}, $page), $params{destpage}));
                        $template->param(title => pagetitle(basename($page)));
                        $template->param(ctime => displaytime($pagectime{$page}));
 
@@ -205,15 +202,17 @@
                }
        
                if ($rss) {
-                       will_render($params{page}, rsspage($params{page}));
-                       writefile(rsspage($params{page}), $config{destdir},
+                       my $rssp=rsspage($params{page});
+                       will_render($params{page}, $rssp);
+                       writefile($rssp, $config{destdir},
                                genfeed("rss", $rssurl, $desc, $params{page}, @list));
                        $toping{$params{page}}=1 unless $config{rebuild};
                        $feedlinks{$params{destpage}}=qq{<link rel="alternate" type="application/rss+xml" title="RSS" href="$rssurl" />};
                }
                if ($atom) {
-                       will_render($params{page}, atompage($params{page}));
-                       writefile(atompage($params{page}), $config{destdir},
+                       my $atomp=atompage($params{page});
+                       will_render($params{page}, $atomp);
+                       writefile($atomp, $config{destdir},
                                genfeed("atom", $atomurl, $desc, $params{page}, @list));
                        $toping{$params{page}}=1 unless $config{rebuild};
                        $feedlinks{$params{destpage}}=qq{<link rel="alternate" type="application/atom+xml" title="Atom" href="$atomurl" />};
@@ -288,16 +287,21 @@
        return $content;
 } #}}}
 
-sub rsspage ($) { #{{{
+sub basepage ($) { #{{{
        my $page=shift;
+       
+       $page=htmlpage($page);
+       $page =~ s/\.html$//;
+       
+       return $page;
+} #}}}
 
-       return $page.".rss";
+sub rsspage ($) { #{{{
+       return targetpage(shift, "rss");
 } #}}}
 
 sub atompage ($) { #{{{
-       my $page=shift;
-
-       return $page.".atom";
+       return targetpage(shift, "atom");
 } #}}}
 
 sub genfeed ($$$$@) { #{{{
Index: IkiWiki/Plugin/aggregate.pm
===================================================================
--- IkiWiki/Plugin/aggregate.pm (revision 2806)
+++ IkiWiki/Plugin/aggregate.pm (working copy)
@@ -320,7 +320,7 @@
                # NB: This doesn't check for path length limits.
                eval q{use POSIX};
                my $max=POSIX::pathconf($config{srcdir}, &POSIX::_PC_NAME_MAX);
-               if (defined $max && length(htmlpage($page)) >= $max) {
+               if (defined $max && length(htmlfn($page)) >= $max) {
                        $c="";
                        $page=$feed->{dir}."/item";
                        while (exists $IkiWiki::pagecase{lc $page.$c} ||
@@ -356,7 +356,7 @@
        if (ref $feed->{tags}) {
                $template->param(tags => [map { tag => $_ }, @{$feed->{tags}}]);
        }
-       writefile(htmlpage($guid->{page}), $config{srcdir},
+       writefile(htmlfn($guid->{page}), $config{srcdir},
                $template->output);
 
        # Set the mtime, this lets the build process get the right creation
@@ -434,4 +434,8 @@
        return "$config{srcdir}/".htmlpage($page);
 } #}}}
 
+sub htmlfn ($) { #{{{
+       return shift().".html";
+} #}}}
+
 1
Index: IkiWiki/Plugin/linkmap.pm
===================================================================
--- IkiWiki/Plugin/linkmap.pm   (revision 2806)
+++ IkiWiki/Plugin/linkmap.pm   (working copy)
@@ -49,9 +49,7 @@
        my %mapitems = ();
        foreach my $item (keys %links) {
                if (pagespec_match($item, $params{pages}, $params{page})) {
-                       my $link=htmlpage($item);
-                       $link=IkiWiki::abs2rel($link, IkiWiki::dirname($params{page}));
-                       $mapitems{$item}=$link;
+                       $mapitems{$item}=urlto($item, $params{destpage});
                }
        }
 
Index: doc/usage.mdwn
===================================================================
--- doc/usage.mdwn      (revision 2806)
+++ doc/usage.mdwn      (working copy)
@@ -244,6 +244,10 @@
 
   Log to syslog(3).
 
+* --usedirs
+
+  Create output files named page/index.html instead of page.html.
+
 * --w3mmode, --no-w3mmode
 
   Enable [[w3mmode]], which allows w3m to use ikiwiki as a local CGI script,
Index: doc/plugins/write.mdwn
===================================================================
--- doc/plugins/write.mdwn      (revision 2806)
+++ doc/plugins/write.mdwn      (working copy)
@@ -412,6 +412,10 @@
 
 This is the standard gettext function, although slightly optimised.
 
+#### `urlto($$)`
+
+Construct a relative url to the first parameter from the second.
+
 ## RCS plugins
 
 ikiwiki's support for revision control systems also uses pluggable perl
Index: doc/ikiwiki.setup
===================================================================
--- doc/ikiwiki.setup   (revision 2806)
+++ doc/ikiwiki.setup   (working copy)
@@ -94,6 +94,8 @@
        syslog => 0,
        # To link to user pages in a subdirectory of the wiki.
        #userdir => "users",
+       # To enable alternate output filenames.
+       #usedirs => 1,
 
        # To add plugins, list them here.
        #add_plugins => [qw{goodstuff openid search wikitext camelcase
</pre>