MoinMoin

Contents

MoinMoin

About

Written in python.
Extremely extensible.
Plaintext data-files.
No database.
It's fast.

Install

   1 aptitude search apache2 python-moinmoin python-xapian

Configure

Apache2 Config

Enable modules

   1 a2enmod ssl wsgi rewrite header

Do the other security stuff from Apache2
A configuration using mod_wsgi:

/etc/apache2/sites-available/wiki.rockstable.it_443.conf

   1 <IfModule mod_ssl.c>
   2         <VirtualHost _default_:443>
   3                 Define          SERVER_NAME wiki.rockstable.it
   4                 ServerName      ${SERVER_NAME}
   5                 ServerAlias     wiki1.rockstable.it
   6         ServerAlias     rockstable.org
   7         ServerAlias     www.rockstable.org
   8                 ServerAdmin     webmaster@rockstable.it
   9 
  10                 DocumentRoot    /var/www/html
  11 
  12                 # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
  13                 # error, crit, alert, emerg.
  14                 # It is also possible to configure the loglevel for particular
  15                 # modules, e.g.
  16                 #LogLevel info ssl:warn
  17 
  18                 ErrorLog ${APACHE_LOG_DIR}/error.log
  19                 CustomLog ${APACHE_LOG_DIR}/access.log combined
  20 
  21                 # For most configuration files from conf-available/, which are
  22                 # enabled or disabled at a global level, it is possible to
  23                 # include a line for only one particular virtual host. For example the
  24                 # following line enables the CGI configuration for this host only
  25                 # after it has been globally disabled with "a2disconf".
  26                 #Include conf-available/serve-cgi-bin.conf
  27 
  28                 #   SSL Engine Switch:
  29                 #   Enable/Disable SSL for this virtual host.
  30                 SSLEngine on
  31 
  32                 #   A self-signed (snakeoil) certificate can be created by installing
  33                 #   the ssl-cert package. See
  34                 #   /usr/share/doc/apache2/README.Debian.gz for more info.
  35                 #   If both key and certificate are stored in the same file, only the
  36                 #   SSLCertificateFile directive is needed.
  37                 #SSLCertificateFile     /etc/ssl/certs/ssl-cert-snakeoil.pem
  38                 #SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
  39                 SSLCertificateFile      /etc/letsencrypt/live/wiki.rockstable.it/fullchain.pem
  40                 SSLCertificateKeyFile   /etc/letsencrypt/live/wiki.rockstable.it/privkey.pem
  41 
  42 
  43                 #   Server Certificate Chain:
  44                 #   Point SSLCertificateChainFile at a file containing the
  45                 #   concatenation of PEM encoded CA certificates which form the
  46                 #   certificate chain for the server certificate. Alternatively
  47                 #   the referenced file can be the same as SSLCertificateFile
  48                 #   when the CA certificates are directly appended to the server
  49                 #   certificate for convinience.
  50                 #SSLCertificateChainFile /etc/apache2/ssl.crt/server-ca.crt
  51 
  52                 #   Certificate Authority (CA):
  53                 #   Set the CA certificate verification path where to find CA
  54                 #   certificates for client authentication or alternatively one
  55                 #   huge file containing all of them (file must be PEM encoded)
  56                 #   Note: Inside SSLCACertificatePath you need hash symlinks
  57                 #                to point to the certificate files. Use the provided
  58                 #                Makefile to update the hash symlinks after changes.
  59                 #SSLCACertificatePath /etc/ssl/certs/
  60                 #SSLCACertificateFile /etc/apache2/ssl.crt/ca-bundle.crt
  61 
  62                 #   Certificate Revocation Lists (CRL):
  63                 #   Set the CA revocation path where to find CA CRLs for client
  64                 #   authentication or alternatively one huge file containing all
  65                 #   of them (file must be PEM encoded)
  66                 #   Note: Inside SSLCARevocationPath you need hash symlinks
  67                 #                to point to the certificate files. Use the provided
  68                 #                Makefile to update the hash symlinks after changes.
  69                 #SSLCARevocationPath /etc/apache2/ssl.crl/
  70                 #SSLCARevocationFile /etc/apache2/ssl.crl/ca-bundle.crl
  71 
  72                 #   Client Authentication (Type):
  73                 #   Client certificate verification type and depth.  Types are
  74                 #   none, optional, require and optional_no_ca.  Depth is a
  75                 #   number which specifies how deeply to verify the certificate
  76                 #   issuer chain before deciding the certificate is not valid.
  77                 #SSLVerifyClient require
  78                 #SSLVerifyDepth  10
  79 
  80                 #   SSL Engine Options:
  81                 #   Set various options for the SSL engine.
  82                 #   o FakeBasicAuth:
  83                 #        Translate the client X.509 into a Basic Authorisation.  This means that
  84                 #        the standard Auth/DBMAuth methods can be used for access control.  The
  85                 #        user name is the `one line' version of the client's X.509 certificate.
  86                 #        Note that no password is obtained from the user. Every entry in the user
  87                 #        file needs this password: `xxj31ZMTZzkVA'.
  88                 #   o ExportCertData:
  89                 #        This exports two additional environment variables: SSL_CLIENT_CERT and
  90                 #        SSL_SERVER_CERT. These contain the PEM-encoded certificates of the
  91                 #        server (always existing) and the client (only existing when client
  92                 #        authentication is used). This can be used to import the certificates
  93                 #        into CGI scripts.
  94                 #   o StdEnvVars:
  95                 #        This exports the standard SSL/TLS related `SSL_*' environment variables.
  96                 #        Per default this exportation is switched off for performance reasons,
  97                 #        because the extraction step is an expensive operation and is usually
  98                 #        useless for serving static content. So one usually enables the
  99                 #        exportation for CGI and SSI requests only.
 100                 #   o OptRenegotiate:
 101                 #        This enables optimized SSL connection renegotiation handling when SSL
 102                 #        directives are used in per-directory context.
 103                 #SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire
 104                 #<FilesMatch "\.(cgi|shtml|phtml|php)$">
 105                 #               SSLOptions +StdEnvVars
 106                 #</FilesMatch>
 107                 #<Directory /usr/lib/cgi-bin>
 108                 #               SSLOptions +StdEnvVars
 109                 #</Directory>
 110 
 111                 #   SSL Protocol Adjustments:
 112                 #   The safe and default but still SSL/TLS standard compliant shutdown
 113                 #   approach is that mod_ssl sends the close notify alert but doesn't wait for
 114                 #   the close notify alert from client. When you need a different shutdown
 115                 #   approach you can use one of the following variables:
 116                 #   o ssl-unclean-shutdown:
 117                 #        This forces an unclean shutdown when the connection is closed, i.e. no
 118                 #        SSL close notify alert is send or allowed to received.  This violates
 119                 #        the SSL/TLS standard but is needed for some brain-dead browsers. Use
 120                 #        this when you receive I/O errors because of the standard approach where
 121                 #        mod_ssl sends the close notify alert.
 122                 #   o ssl-accurate-shutdown:
 123                 #        This forces an accurate shutdown when the connection is closed, i.e. a
 124                 #        SSL close notify alert is send and mod_ssl waits for the close notify
 125                 #        alert of the client. This is 100% SSL/TLS standard compliant, but in
 126                 #        practice often causes hanging connections with brain-dead browsers. Use
 127                 #        this only for browsers where you know that their SSL implementation
 128                 #        works correctly.
 129                 #   Notice: Most problems of broken clients are also related to the HTTP
 130                 #   keep-alive facility, so you usually additionally want to disable
 131                 #   keep-alive for those clients, too. Use variable "nokeepalive" for this.
 132                 #   Similarly, one has to force some clients to use HTTP/1.0 to workaround
 133                 #   their broken HTTP/1.1 implementation. Use variables "downgrade-1.0" and
 134                 #   "force-response-1.0" for this.
 135                 # BrowserMatch "MSIE [2-6]" \
 136                 #               nokeepalive ssl-unclean-shutdown \
 137                 #               downgrade-1.0 force-response-1.0
 138 
 139                 #RewriteCond %{REQUEST_URI}      !=/server-status
 140                 RewriteEngine on
 141                 RewriteCond %{HTTP_HOST}                !=${SERVER_NAME}
 142                 RewriteRule ^/(.*)$             https://${SERVER_NAME}/$1 [R=301,L]
 143 
 144                 ### VERY OLD EDITOR :-(
 145                 AliasMatch "^/moin_static[0-9]*/applets/FCKeditor/(.*)" "/usr/share/fckeditor/$1"
 146                 <Directory "/usr/share/fckeditor/">
 147                         Options None
 148                         AllowOverride None
 149                 </Directory>
 150 
 151                 ### Serve static contents (images, javascript, css...) ###
 152                 # The path to static contents changes (named after moinmoin version).
 153                 AliasMatch "^/moin_static[0-9]*/(.*)" "/usr/share/moin/htdocs/$1"
 154                 <Directory "/usr/share/moin/htdocs/">
 155                         #Options                -Indexes -FollowSymlinks
 156                         Options                 -Indexes
 157                         AllowOverride   None
 158                 </Directory>
 159 
 160                 AliasMatch "^/qr/([0-9a-f]+\.(png|svg))" "/srv/moin/rs_admin/data/qr/$1"
 161                 <Directory "/srv/moin/rs_admin/data/qr">
 162                         Options                 -Indexes -FollowSymlinks
 163                         Require                 all granted
 164                         AllowOverride   None
 165                 </Directory>
 166 
 167                 #<Directory "/opt/moinmoin-memodump/memodump">
 168                 #       #Options -Indexes -FollowSymlinks
 169                 #       Options -Indexes
 170                 #       AllowOverride None
 171                 #</Directory>
 172 
 173                 ### vhost ###
 174                 Alias /robots.txt       /usr/share/moin/htdocs/robots.txt
 175                 Alias /favicon.ico  /usr/share/moin/htdocs/favicon.ico
 176 
 177                 WSGIDaemonProcess rs_admin user=moin-rs_admin group=moin-rs_admin \
 178                         processes=5 threads=10 maximum-requests=1000 umask=0007 \
 179                         display-name=moin-wsgi-rs_admin
 180                 WSGIProcessGroup rs_admin
 181                 WSGIScriptAlias / "/usr/share/moin/server/moin.wsgi"
 182 
 183                 # Always ensure Cookies have "Secure" set (JAH 2012/1)
 184                 Header edit Set-Cookie (?i)^(.*)(;\s*secure)??((\s*;)?(.*)) "$1; Secure$3$4"
 185 
 186         </VirtualHost>
 187 </IfModule>
 188 
 189 # vim: syntax=apache ts=4 sw=4 sts=4 sr noet

   1 a2ensite wiki.rockstable.it_443
   2 # APACHE2 won't come up at this stage (missing users)
   3 systemctl restart apache2.service

Farmconfig

After changes to farmconfig.py or wikiconfig.py apache2 has to be restarted.
The WSGI-Script /usr/share/moin/server/moin.wsgi adds /etc/moin to syspath and performs it's configuration like this.

/etc/moin/farmconfig.py

   1 # a2) Path of the directory where wikiconfig.py / farmconfig.py is located.
   2 #     See wiki/config/... for some sample config files.
   3 #sys.path.insert(0, '/etc/moin')
   4 sys.path.insert(0, '/etc/moin')

/etc/moin/farmconfig.py

   1 # -*- coding: iso-8859-1 -*-
   2 # IMPORTANT! This encoding (charset) setting MUST be correct! If you live in a
   3 # western country and you don't know that you use utf-8, you probably want to
   4 # use iso-8859-1 (or some other iso charset). If you use utf-8 (a Unicode
   5 # encoding) you MUST use: coding: utf-8
   6 # That setting must match the encoding your editor uses when you modify the
   7 # settings below. If it does not, special non-ASCII chars will be wrong.
   8 
   9 """
  10     MoinMoin - Configuration for a wiki farm
  11 94
  12     If you run a single wiki only, you can keep the "wikis" list "as is"
  13     (it has a single rule mapping all requests to mywiki.py).
  14 
  15     Note that there are more config options than you'll find in
  16     the version of this file that is installed by default; see
  17     the module MoinMoin.config.multiconfig for a full list of names and their
  18     default values.
  19 
  20     Also, the URL http://moinmo.in/HelpOnConfiguration has
  21     a list of config options.
  22 """
  23 
  24 
  25 # Wikis in your farm --------------------------------------------------
  26 
  27 # If you run multiple wikis, you need this list of pairs (wikiname, url
  28 # regular expression). moin processes that list and tries to match the
  29 # regular expression against the URL of this request - until it matches.
  30 # Then it loads the <wikiname>.py config for handling that request.
  31 
  32 # Important:
  33 #  * the left part is the wikiname enclosed in double quotes
  34 #  * the left part must be a valid python module name, so better use only
  35 #    lower letters "a-z" and "_". Do not use blanks, dots or minus there!
  36 #    E.g. use "foo_bar_org", NOT: "foo-bar.org"!
  37 #  * the right part is the url re, use r"..." for it
  38 #  * in the right part ".*" means "everything". Just "*" does not work like
  39 #    for filenames on the shell / commandline, you must use ".*" as it is a RE.
  40 #  * in the right part, "^" means "beginning" and "$" means "end"
  41 
  42 wikis = [
  43 
  44     # wikiname, url regular expression
  45     # ---------------------------------------------------------------
  46     #("mywiki", r".*"),   # this is ok for a single wiki
  47     ("rs_admin", r"^https?://wiki\.rockstable\.it/.*$"),   # this is ok for a single wiki
  48 
  49     # for multiple wikis, do something like this:
  50     #("wiki1", r"^http://wiki1\.example\.org/.*$"),
  51     #("wiki2", r"^https?://wiki2\.example\.org/.*$"),
  52 ]
  53 
  54 
  55 # Common configuration for all wikis ----------------------------------
  56 
  57 # Everything that should be configured the same way should go here,
  58 # anything else that should be different should go to the single wiki's
  59 # config.
  60 # In that single wiki's config, we will use the class FarmConfig we define
  61 # below as the base config settings and only override what's different.
  62 #
  63 # In exactly the same way, we first include MoinMoin's Config Defaults here -
  64 # this is to get everything to sane defaults, so we need to change only what
  65 # we like to have different:
  66 
  67 from MoinMoin.config import multiconfig, url_prefix_static
  68 
  69 # Now we subclass this DefaultConfig. This means that we inherit every setting
  70 # from the DefaultConfig, except those we explicitely define different.
  71 
  72 class FarmConfig(multiconfig.DefaultConfig):
  73 
  74     # Critical setup  ---------------------------------------------------
  75 
  76     # The URL prefix we use to access the static stuff (img, css, js).
  77     # Note: moin runs a static file server at url_prefix_static path (relative
  78     # to the script url).
  79     # If you run your wiki script at the root of your site (/), just do NOT
  80     # use this setting and it will automatically work.
  81     # If you run your wiki script at /mywiki, you need to use this:
  82     #url_prefix_static = '/mywiki' + url_prefix_static
  83     # If you need different url_prefix_static setups for your wikis,
  84     # you'll have to do it in each wiki's config.
  85 
  86     # Security ----------------------------------------------------------
  87 
  88     # This is checked by some rather critical and potentially harmful actions,
  89     # like despam or PackageInstaller action:
  90     superuser = [u"RockStable", ]
  91 
  92     ### DISABLE USER ACCOUNT CREATION -> SPAMMERS
  93     #actions_excluded = ['newaccount']
  94 
  95     # IMPORTANT: grant yourself admin rights! replace YourName with
  96     # your user name. See HelpOnAccessControlLists for more help.
  97     # All acl_rights_xxx options must use unicode [Unicode]
  98     #acl_rights_before = u"YourName:read,write,delete,revert,admin"
  99     acl_hierarchic       = True
 100     acl_rights_default  = u"ContribGroup:read,write,revert All:read"
 101     acl_rights_before   = u"RockStable:read,write,delete,revert,admin AdminGroup:read,write,revert,delete"
 102     acl_rights_after    = u"BadGuysGroup:"
 103 
 104     # Link spam protection for public wikis (uncomment to enable).
 105     # Needs a reliable internet connection.
 106     #from MoinMoin.security.antispam import SecurityPolicy
 107 
 108 
 109     # Mail --------------------------------------------------------------
 110 
 111     # Configure to enable subscribing to pages (disabled by default) or
 112     # sending forgotten passwords.
 113 
 114     # SMTP server, e.g. "mail.provider.com" (empty or None to disable mail)
 115     #mail_smarthost = ""
 116     mail_smarthost = "localhost"
 117 
 118     # The return address, e.g u"Jürgen Wiki <noreply@mywiki.org>" [Unicode]
 119     #mail_from = u""
 120 
 121     # "user pwd" if you need to use SMTP AUTH
 122     #mail_login = ""
 123 
 124 
 125     # User interface ----------------------------------------------------
 126 
 127     # Add your wikis important pages at the end. It is not recommended to
 128     # remove the default links.  Leave room for user links - don't use
 129     # more than 6 short items.
 130     # You MUST use Unicode strings here, but you need not use localized
 131     # page names for system and help pages, those will be used automatically
 132     # according to the user selected language. [Unicode]
 133     navi_bar = [
 134         # If you want to show your page_front_page here:
 135         #u'%(page_front_page)s',
 136         u'RecentChanges',
 137         u'FindPage',
 138         u'HelpContents',
 139     ]
 140 
 141     # The default theme anonymous or new users get
 142     theme_default = 'modernized'
 143 
 144 
 145     # Language options --------------------------------------------------
 146 
 147     # See http://moinmo.in/ConfigMarket for configuration in
 148     # YOUR language that other people contributed.
 149 
 150     # The main wiki language, set the direction of the wiki pages
 151     language_default = 'en'
 152     tz_offset = 1.0 # default time zone offset in hours from UTC    
 153 
 154     # the following regexes should match the complete name when used in free text
 155     # the group 'all' shall match all, while the group 'key' shall match the key only
 156     # e.g. CategoryFoo -> group 'all' ==  CategoryFoo, group 'key' == Foo
 157     # moin's code will add ^ / $ at beginning / end when needed
 158     # You must use Unicode strings here [Unicode]
 159     page_category_regex = ur'(?P<all>Category(?P<key>\S+))'
 160     page_dict_regex = ur'(?P<all>(?P<key>\S+)Dict)'
 161     page_group_regex = ur'(?P<all>(?P<key>\S+)Group)'
 162     page_template_regex = ur'(?P<all>(?P<key>\S+)Template)'
 163 
 164     # Content options ---------------------------------------------------
 165 
 166     # Show users hostnames in RecentChanges
 167     show_hosts = True
 168 
 169     # Show the interwiki name (and link it to page_front_page) in the Theme,
 170     # nice for farm setups or when your logo does not show the wiki's name.
 171     show_interwiki = 1
 172     #logo_string = u''
 173     logo_string = u'<img src="%s/common/rockstable.gif" alt="wiki.rockstable.it">' % url_prefix_static
 174 
 175     # Enable graphical charts, requires gdchart.
 176     #chart_options = {'width': 600, 'height': 300}
 177 
 178     page_credits        = [
 179         '<a href="http://moinmo.in/" title="This site uses the MoinMoin Wiki software.">MoinMoin Powered</a>',
 180         '<a href="http://moinmo.in/Python" title="MoinMoin is written in Python.">Python Powered</a>',
 181         '<a href="http://moinmo.in/GPL" title="MoinMoin is GPL licensed.">GPL licensed</a>',
 182         '<a href="http://validator.w3.org/check?uri=referer" title="Click here to validate this page.">Valid HTML 4.01</a>',
 183         '<a href="https://github.com/dossist/moinmoin-memodump">Theme Memodump</a>'
 184         ]
 185 
 186     #XAPIAN-SEARCH
 187     show_version = False # show moin's version at the bottom of a page
 188     show_timings = False # show some timing values at bottom of a page
 189     xapian_index_dir = '/srv/moin/rs_admin/data/cache/xapian' # Directory where the Xapian search indexex are stored
 190     xapian_index_history = False # Index old versions of page
 191     xapian_search = True # True to enable the fast, indexed search (based on the Xapian search library)
 192     xapian_stemming = True # True to enable Xapian word stemmer usage for indexing / searching
 193 
 194     ### TEXTCHAS
 195     # members of this don't get textchas
 196     #textchas_disabled_group = [ u"ContribGroup", u"FamilyGroup", u"DownloadGroup", u"PulsGroup" ]
 197     textchas_disabled_group = u"TextChasDisabledGroup"
 198     textchas_disabled_group = u"TrustedEditorGroup" # members of this don't get textchas
 199     textchas = {
 200         'en': { # silly english example textchas (do not use them!)
 201             u"Enter the first 9 digits of Pi.": ur"3\.14159265",
 202             u"What is the opposite of 'day'?": ur"(night|nite)",
 203             # ...
 204         },
 205         'de': { # some german textchas
 206             u"Gib die ersten 9 Stellen von Pi ein.": ur"3\.14159265",
 207             u"Was ist das Gegenteil von 'Tag'?": ur"[Nn]acht",
 208             # ...
 209         },
 210         # you can add more languages if you like
 211     }

Moin Instances

Decisions

I decided to:

host my moin instances from /srv/moin/.
- This keeps /usr/share/moin clean and untouched.
- Separates moin variable data from distribution files and home-directories.
give every moin instance a separate system user
- which home-directories are furthermore separated in /srv/moin from those of "casual" users.
WSGI-Daemon-User must neither be able to write any config or code (like plugins).

Wikiconfig

The "wikiconfig.py" inherits from farmconfig and overrides the "Default" values.

   1 # -*- coding: iso-8859-1 -*-   
   2 # IMPORTANT! This encoding (charset) setting MUST be correct! If you live in a
   3 # western country and you don't know that you use utf-8, you probably want to
   4 # use iso-8859-1 (or some other iso charset). If you use utf-8 (a Unicode
   5 # encoding) you MUST use: coding: utf-8
   6 # That setting must match the encoding your editor uses when you modify the
   7 # settings below. If it does not, special non-ASCII chars will be wrong.
   8 
   9 """
  10 This is a sample config for a wiki that is part of a wiki farm and uses
  11 farmconfig for common stuff. Here we define what has to be different from
  12 the farm's common settings.
  13 """
  14 
  15 # we import the FarmConfig class for common defaults of our wikis:
  16 from farmconfig import FarmConfig
  17 
  18 # now we subclass that config (inherit from it) and change what's different:
  19 class Config(FarmConfig):
  20 
  21     # basic options (you normally need to change these)
  22     sitename            = u'Rockstable Wiki' # [Unicode]
  23     interwikiname       = u'Rockstable Wiki' # [Unicode]
  24 
  25     # name of entry page / front page [Unicode], choose one of those:
  26 
  27     # a) if most wiki content is in a single language
  28     page_front_page     = u"RockstableWiki"
  29 
  30     # b) if wiki content is maintained in many languages
  31     #page_front_page = u"FrontPage" 
  32 
  33     #data_dir           = '/usr/share/moin/rs_admin/data'
  34     #data_underlay_dir  = '/usr/share/moin/rs_admin/underlay'
  35 
  36     data_dir            = '/srv/moin/rs_admin/data'
  37     data_underlay_dir   = '/srv/moin/rs_admin/underlay'
  38     theme_default       = 'memodump'
  39         
  40     acl_hierarchic      = True
  41     acl_rights_default  = u"ContribGroup:read,write,revert All:read"
  42     acl_rights_before   = u"RockStable:read,write,delete,revert,admin AdminGroup:read,write,revert,delete"
  43     acl_rights_after    = u"BadGuysGroup:"
  44     
  45     allowed_actions     = ['AttachFile']
  46 
  47     mail_from = u"Noreply Rockstable Wiki <noreply@wiki.rockstable.it>"
  48 
  49     #edit_bar = ['Edit', 'Comments', 'Discussion', 'Info', 'Subscribe', 'Quicklink', 'Attachments', 'ActionsMenu']
  50     editor_default = 'text' # which editor is called when nothing is specified
  51     editor_force = False # force using the default editor
  52     #editor_ui = 'freechoice' # which editor links are shown on user interface
  53     #edit_locking = 'warn 10' # None, 'warn <timeout mins>', 'lock <timeout mins>'
  54     #edit_rows = 20
  55     
  56     #surge_action_limits = None # disable surge protection
  57     surge_action_limits = { # allow max. <count> <action> requests per <dt> secs
  58         # action: (count, dt)
  59         'show': (20, 60),
  60         'raw': (20, 40),  # some people use this for css
  61         'AttachFile': (60, 60),
  62         'diff': (30, 60),
  63         'fullsearch': (5, 60),
  64         'edit': (60, 120),
  65         'rss_rc': (1, 60),
  66         'default': (30, 60),
  67     }
  68     surge_lockout_time = 300 # secs you get locked out when you ignore warnings

Create groups and users

   1 addgroup --system moin_instance
   2 addgroup --system moin-rs_admin
   3 adduser --system \
   4         --ingroup moin-rs_admin \
   5         --home /srv/moin/rs_admin \
   6         --gecos "MoinMoin WSGI Rockstable Admin Wiki,," \
   7         moin-rs_admin
   8 adduser moin-rs_admin moin_instance

Instance Directories

Prepare data and underlay directory

   1 cp -p /usr/share/moin/underlay /srv/moin/rs_admin/underlay
   2 cp -p /usr/share/moin/data /srv/moin/rs_admin/data

Maybe a create a "reverse" link for those who expect instance directory in /usr/share/moin as a hint.

   1 ln -s /srv/moin/rs_admin /usr/share/moin

Permissions

Moinmoin is very sensitive to problems with unix-permissions and will quit with an error. If you are lucky moinmoin even shows the path on the website.

Following steps are subsequential and should all be done. Make sure everything in instance directory is owned by the wsgi-daemon user:

   1 chown -R moin-rs_admin /srv/moin/rs_admin/{data,underlay}
   2 find /srv/moin/rs_admin/underlay -type d -exec chmod 750 {} \;
   3 find /srv/moin/rs_admin/data -type d -exec chmod 750 {} \;

Disallow the daemon to write its code (e.g. plugins)

   1 find /srv/moin/rs_admin/data/{plugin,wiki} -type d \
   2        -exec chmod 750 {} \; \
   3        -exec chown 0:moin-rs_admin {} \;
   4 find /srv/moin/rs_admin/data/{plugin,wiki} -type f \
   5        -exec chmod 640 {} \; \
   6        -exec chown 0:moin-rs_admin {} \;

Make farmconfig and wikiconfig readable via group permissions.

   1 ### 2. DISSALLOW THE DAEMON TO WRITE ITS CONFIG
   2 chgrp moin_instance /etc/moin/farmconfig.py
   3 chgrp moin-rs_admin /etc/moin/rs_admin.py

Restart apache2 to adjust group-memberships of wsgi process.

   1 systemctl restart apache2

Trouble-Shooting

If you have moved directories or migrated the wiki to a new version, issue the next commands. Or you'll see 501 - Internal Server Error

   1 sudo -u moin-rs_admin -g moin-rs_admin -- moin \
   2         --config-dir /srv/moin/rs_admin/data \
   3         --wiki-url="https://wiki.rockstable.it/" \
   4         maint cleancache
   5 sudo -u moin-rs_admin -g moin-rs_admin -- moin \
   6         --config-dir /srv/moin/rs_admin/data \
   7         --wiki-url="https://wiki.rockstable.it/" \
   8         migration data
   9 sudo -u moin-rs_admin -g moin-rs_admin -- moin \
  10         --config-dir /srv/moin/rs_admin/data \
  11         --wiki-url="https://wiki.rockstable.it/"
  12         maint makecache
  13 find /srv/moin/rs_admin/data \
  14         -type f \!  -user moin-rs_admin \
  15         -exec chown moin-rs_admin:moin-rs_admin {} \;

Spammers and Griefers

Shame on you!

Your site should be protected from these guys!

Spammers and Griefers automatically

create a new account and
use it to create a new page or deface unprotected sites.

There only a few means of protection against them:

Restrictive ACLs
Captchas (completely automated public Turing test to tell computers and humans apart)
Textchas
Disabling account registration, entirely and e.g. setup an ldap-backend.
Patching MoinMoin code to change the way accounts are created.
Be creative …

TextChas

As a supplementary i decided to setup TextChas, which are also used during registration A localized question is display on top of the editor, that wants to be answered on submission.

Here's an example that can be embedded in your wikiconfig.py

   1     textchas_disabled_group = u"TrustedEditorGroup" # members of this don't get textchas
   2     textchas = {
   3         'en': { # silly english example textchas (do not use them!)
   4             u"Enter the first 9 digits of Pi.": ur"3\.14159265",
   5             u"What is the opposite of 'day'?": ur"(night|nite)",
   6             # ...
   7         },
   8         'de': { # some german textchas
   9             u"Gib die ersten 9 Stellen von Pi ein.": ur"3\.14159265",
  10             u"Was ist das Gegenteil von 'Tag'?": ur"[Nn]acht",
  11             # ...
  12         },
  13         # you can add more languages if you like
  14     }

Don't forget to setup your textchas_disabled_group, because it's a bit irritating.

Disable Account Creation

/etc/moin/farmconfig.py

   1 ### DISABLE USER ACCOUNT CREATION -> SPAMMERS
   2 actions_excluded = ['newaccount']

Free wiki from bot accounts

Interate through moinmoin users and decide if hot or not. If it's a spammer it's moved to a subfolder.

   1 #!/bin/bash
   2 
   3 DIR_BASE="/srv/moin/rs_admin"
   4 DIR_USER="$DIR_BASE/data/user"
   5 DIR_SPAM="$DIR_USER/spammer"
   6 SPAMMER_COUNT=0
   7 USER_COUNT=0
   8 
   9 [ -d "$DIR_SPAM" ] || mkdir "$DIR_SPAM"
  10 
  11 stats () {
  12         cat <<-EOHERE
  13 
  14         SPAMMER_COUNT:  $SPAMMER_COUNT
  15         USER_COUNT:     $USER_COUNT
  16         EOHERE
  17 }
  18 
  19 for USER in $(find "$DIR_USER" \
  20         -maxdepth 1 -type f \
  21         -regextype posix-extended \
  22         -regex "$DIR_USER/[0-9]+\.[0-9]+\.[0-9]+"); do
  23         grep '^email=' "$USER"|cut -f2 -d=
  24         ANSWER=n
  25         read -n1 -p "Spammer [y/N/q]: " ANSWER
  26         if [ "$ANSWER" = "y" ]; then
  27                 echo
  28                 mv -v "$USER" "$DIR_SPAM"
  29                 ((SPAMMER_COUNT++))
  30         elif [ "$ANSWER" = "q" ]; then
  31                 echo "Exiting…"
  32                 stats
  33                 exit
  34         else
  35                 ((USER_COUNT++))
  36         fi
  37         echo
  38 done
  39 
  40 stats

Search engine optimization - SEO

#MOBILE FIRST CSS adjustments
- Test it https://search.google.com/test/mobile-friendly
Add a sitemap to google search console
Just append ?action=sitemap to a URL and register it in Google Search Console https://wiki.rockstable.it/RockstableWiki?action=sitemap

Maintainance

cleancache

This script allows you to globally delete all the cache files in the directories:

<data_dir>/pages/PageName/cache/
<data_dir>/cache
<user_dir>/cache

Please see * https://moinmo.in/MoinCaching

   1 ### DISPLAY CLI OPTIONS
   2 sudo -u moin-rs_admin -g moin-rs_admin -- moin \
   3         --config-dir="/etc/moin" \
   4         --wiki-url="https://wiki.rockstable.it/" \
   5         maint cleancache --help
   6 ### PERFORM CLEANUP
   7 sudo -u moin-rs_admin -g moin-rs_admin -- moin \
   8         --config-dir="/etc/moin" \
   9         --wiki-url="https://wiki.rockstable.it/" \
  10         maint cleancache
  11 2022-11-27 14:20:10,833 INFO MoinMoin.log:151 using logging configuration read from built-in fallback in MoinMoin.log module
  12 2022-11-27 14:20:10,833 INFO MoinMoin.log:157 Running MoinMoin 1.9.9 release code from /usr/lib/python2.7/dist-packages/MoinMoin
  13 2022-11-27 14:20:10,901 INFO MoinMoin.config.multiconfig:93 using farm config: /etc/moin/farmconfig.py
  14 2022-11-27 14:20:10,908 INFO MoinMoin.config.multiconfig:127 using wiki config: /etc/moin/rs_admin.py

makecache

This script allows you to create cache files in the directories:

data/pages/PageName/cache
data/cache

Create new cache

   1 ### DISPLAY CLI OPTIONS
   2 sudo -u moin-rs_admin -g moin-rs_admin -- moin \
   3         --config-dir="/etc/moin" \
   4         --wiki-url="https://wiki.rockstable.it/" \
   5         maint makecache --help
   6 
   7 ### CREATE CACHE
   8 sudo -u moin-rs_admin -g moin-rs_admin -- moin \
   9         --config-dir="/etc/moin" \
  10         --wiki-url="https://wiki.rockstable.it/" \
  11         maint makecache      
  12 2022-11-27 14:27:23,025 INFO MoinMoin.log:151 using logging configuration read from built-in fallback in MoinMoin.log module
  13 2022-11-27 14:27:23,025 INFO MoinMoin.log:157 Running MoinMoin 1.9.9 release code from /usr/lib/python2.7/dist-packages/MoinMoin
  14 2022-11-27 14:27:23,093 INFO MoinMoin.config.multiconfig:93 using farm config: /etc/moin/farmconfig.py
  15 2022-11-27 14:27:23,100 INFO MoinMoin.config.multiconfig:127 using wiki config: /etc/moin/rs_admin.py
  16 [TRACE] attachment html: 
  17 [TRACE] attachment html:

cleansessions

This script allows you to clean up session files (usually used to maintain a "logged-in session" for http(s) or xmlrpc).

Delete sessions

   1 ### DISPLAY CLI OPTIONS
   2 sudo -u moin-rs_admin -g moin-rs_admin -- moin \
   3         --config-dir="/etc/moin" \
   4         --wiki-url="https://wiki.rockstable.it/" \
   5         maint cleansessions --help
   6 ### REMOVE OUTDATED SESSIONS
   7 sudo -u moin-rs_admin -g moin-rs_admin -- moin \
   8         --config-dir="/etc/moin" \
   9         --wiki-url="https://wiki.rockstable.it/" \
  10         maint cleansessions
  11 2022-11-27 14:34:25,111 INFO MoinMoin.log:151 using logging configuration read from built-in fallback in MoinMoin.log module
  12 2022-11-27 14:34:25,111 INFO MoinMoin.log:157 Running MoinMoin 1.9.9 release code from /usr/lib/python2.7/dist-packages/MoinMoin
  13 2022-11-27 14:34:25,177 INFO MoinMoin.config.multiconfig:93 using farm config: /etc/moin/farmconfig.py
  14 2022-11-27 14:34:25,184 INFO MoinMoin.config.multiconfig:127 using wiki config: /etc/moin/rs_admin.py
  15 ### REMOVE ALL SESSIONS (LOGOUT EVERYBODY)
  16 sudo -u moin-rs_admin -g moin-rs_admin -- moin \
  17         --config-dir /etc/moin \
  18         --wiki-url="https://wiki.rockstable.it/" \
  19         maint cleansessions --all
  20 2022-11-27 14:44:58,995 INFO MoinMoin.log:151 using logging configuration read from built-in fallback in MoinMoin.log module
  21 2022-11-27 14:44:58,995 INFO MoinMoin.log:157 Running MoinMoin 1.9.9 release code from /usr/lib/python2.7/dist-packages/MoinMoin
  22 2022-11-27 14:44:59,071 INFO MoinMoin.config.multiconfig:93 using farm config: /etc/moin/farmconfig.py
  23 2022-11-27 14:44:59,078 INFO MoinMoin.config.multiconfig:127 using wiki config: /etc/moin/rs_admin.py

cleanpage

This tool outputs a shell script, which upon execution will remove unused or trashed pages from the wiki.

   1 ### DISPLAY CLI OPTIONS
   2 sudo -u moin-rs_admin -g moin-rs_admin -- moin \
   3         --config-dir="/etc/moin" \
   4         --wiki-url="https://wiki.rockstable.it/" \
   5         maint cleanpage --help
   6 sudo -u moin-rs_admin -g moin-rs_admin -- moin \
   7         --config-dir="/etc/moin" \
   8         --wiki-url="https://wiki.rockstable.it/" \
   9         maint cleanpage \
  10         |tee /tmp/cleanpage.sh
  11 ### VALIDATE AND ADJUST SCRIPT
  12 sed -r 's/^# ok: /#ok: /;
  13         s/ # (\w+)$//' \
  14         /tmp/cleanpage.sh \
  15         |grep -v '^#ok'\
  16         |column -t
  17 view /tmp/cleanpage.sh
  18 ### EXECUTE SCRIPT
  19 DATA_DIR="/srv/moin/rs_admin/data"
  20 cd "$DATA_DIR"
  21 mkdir trash deleted
  22 # bash -v /tmp/cleanpage.sh
  23

Lots of pages less.

Customization

Theme Memodump

https://github.com/dossist/moinmoin-memodump

   1 cd /opt
   2 git clone https://github.com/dossist/moinmoin-memodump.git
   3 ln -s /opt/moinmoin-memodump/memodump /usr/share/moin/htdocs/memodump
   4 ln -s /opt/moinmoin-memodump/memodump.py /usr/share/moin/rs_admin/data/plugin/theme

MOBILE FIRST CSS Adjustments

Increase global line-height
memodump/css/bootstrap.css

   1 body {
   2   font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
   3   font-size: 14px;
   4   /* line-height: 1.42857143; */
   5   line-height: 1.5;
   6   color: #333;
   7   background-color: #fff;
   8 }

Increase line-height in (un)ordered lists
memodump/css/bootstrap.css
```
   1 ul,
   2 ol {
   3   margin-top: 0;
   4   margin-bottom: 10px;
   5   line-height: 2;
   6 }
```

Increase container width from 975px to 1200px.

memodump/css/memodump.css

   1 #contentbox {
   2     #width: 975px; /* .container width (1170px) - #sidebar width (195px) */
   3     width: 1200px; /* .container width (1170px) - #sidebar width (195px) */
   4     display: inline-block;
   5     vertical-align: top;
   6 }

Decrease padding of sidebar and contextbox.

memodump/css/memodump.css

   1 #sidebar, #contentbox {
   2     /*padding-left: 15px;
   3     padding-right: 15px;*/
   4     padding-left: 1vw;
   5     padding-right: 1vw;
   6 }
   7  * Decrease padding of codearea
   8  {{{memodump/css/memodump.css

   1 div.codearea pre { /* the pre has no border and is inside the div */
   2     margin: 0;
   3     /* padding: 10pt; */
   4     padding: 1ex;
   5     border: none;
   6 }

Allow (brutal) wrapping of
- words,
  ###
```
   1 tt:not(:empty) { /* not to mess up empty backticks used in order to prevent autolinks */
   2     /* from Bootstrap <code> */
   3     font-family: monospace, monospace;
   4     padding: 2px 4px;
   5     font-size: 90%;
   6     color: #c7254e;
   7     background-color: #f9f2f4;
   8     border-radius: 4px;
   9     word-wrap: break-word;
  10 }
```
- links,
  memodump/css/moinizer.css
```
   1 a.http,
   2 a.https,
   3 a.www {
   4        word-wrap: break-word;
   5 }
```
- and preformatted text.
```
   1 
```

table-of-contents: Disable max-width (and increase line-height to at least 1.7)

   1 #pagebox div.table-of-contents {
   2     font-size: 88%;
   3     text-align: left;
   4     margin: 0.5em 0 0.5em 1em;
   5     /* max-width: 50%; */
   6     /* line-height: 1.7; */
   7     display: table;
   8     /* Bootstrap .well and .well-sm */
   9     min-height: 20px;
  10     padding: 9px;
  11     margin-bottom: 20px;
  12     background-color: #f5f5f5;
  13     border: 1px solid #e3e3e3;
  14     border-radius: 3px;
  15     -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, .05);
  16             box-shadow: inset 0 1px 1px rgba(0, 0, 0, .05);
  17 }

Minify css

https://github.com/zacharyvoase/cssmin

Well, python is installed, so i use python (45KiB).

Install

   1 aptitude install cssmin

Minify

   1 aptitude install cssmin
   2 cd /opt/moinmoin-memodump/memodump/css
   3 FILE="bootstrap.css"
   4 FILE_MIN="${FILE%.css}.min.css"
   5 [ -f "$FILE_MIN" ] && \
   6         mv "$FILE_MIN" "$FILE_MIN.$(date +%F)"
   7 cssmin < "$FILE" > "$FILE_MIN"

Uglify js

StackExchange Unix - How to minify Javascript and CSS with command-line using minify tool?

Install

   1 aptitude install uglifyjs
   2 #aptitude install uglifyjs -t sid
   3

Uglify

   1 FILE="jquery-3.6.0.js"
   2 uglifyjs --webkit --mangle --compress \
   3         -o "${FILE%.js}.min.js" "$FILE"
   4 ls -l "${FILE%.js}"*

Calculate Sub-Resource Integrity Hash

With openssl

   1 FILE="jquery-3.6.0.min.js"
   2 HASH_ALGO="sha256"
   3 HASH="$(openssl dgst -"$HASH_ALGO" -binary "$FILE" \
   4         |openssl base64 -A)"
   5 echo "integrity=\"$HASH_ALGO-$HASH\""

With other standard tools

   1 FILE="jquery-3.6.0.min.js"
   2 HASH_ALGO="256"
   3 HASH="$(shasum -b -a "$HASH_ALGO" $FILE \
   4         |awk '{ print $1 }' \
   5         |xxd -r -p \
   6         |base64)"
   7 echo "integrity=\"sha$HASH_ALGO-$HASH\""

Xapian Search/Index

Xapian Search/Index - create and enable

Add Xapian index

   1 ### BUILD NEW INDEX
   2 sudo -u moin-rs_admin -g moin-rs_admin -- moin \
   3         --config-dir /etc/moin \
   4         --wiki-url="https://wiki.rockstable.it/" \
   5         index build --mode=buildnewindex
   6 ### USE NEW INDEX
   7 sudo -u moin-rs_admin -g moin-rs_admin -- moin \
   8         --config-dir /etc/moin \
   9         --wiki-url="https://wiki.rockstable.it/" \
  10         index build --mode=usenewindex

Xapian Search/Index - rebuild

If you change xapian_ options in wikiconfig you have to user --mode=rebuild

   1 sudo -u moin-rs_admin -g moin-rs_admin -- moin \
   2         --config-dir /etc/moin \
   3         --wiki-url="https://wiki.rockstable.it/" \
   4         index build --mode=rebuild
   5 2022-11-27 14:38:04,636 INFO MoinMoin.log:151 using logging configuration read from built-in fallback in MoinMoin.log module
   6 2022-11-27 14:38:04,636 INFO MoinMoin.log:157 Running MoinMoin 1.9.9 release code from /usr/lib/python2.7/dist-packages/MoinMoin
   7 2022-11-27 14:38:04,729 INFO MoinMoin.config.multiconfig:93 using farm config: /etc/moin/farmconfig.py
   8 2022-11-27 14:38:04,739 INFO MoinMoin.config.multiconfig:127 using wiki config: /etc/moin/rs_admin.py
   9 2022-11-27 14:38:04,793 INFO MoinMoin.search.Xapian.indexing:580 indexing 248 pages...
  10 2022-11-27 14:38:09,906 INFO MoinMoin.search.builtin:214 indexing completed successfully in 5.14 seconds.

Doc is in the Source: /usr/lib/python2.7/dist-packages/MoinMoin/script/index/build.py
Check if it worked on macro <<systeminfo>>

QR-Codes

All Credits and Thanks to: Daniil Alexeyevsky, 2013
https://moinmo.in/MacroMarket/QR

Install qrencode

   1 aptitude install qrencode

This is a slightly modified version of the plugin:

uses SVGs with a higher ECLevel
modifies Unix-Permissions of the generated QRCode in the filesystem

Create macro-file: /srv/moin/rs_admin/data/plugin/macro/QR.py

   1 import os
   2 from subprocess import Popen, PIPE
   3 from hashlib import sha1
   4 from MoinMoin.Page import Page
   5 
   6 qr_options = ["-s", "4", "-t", "svg", "-l", "M"]
   7 
   8 def svg_path(request, pagename, text):
   9     return "/srv/moin/rs_admin/data/qr", svg_filename(text)
  10 def svg_url(request, pagename, text):
  11         return "/qr/" + svg_filename(text)
  12 def svg_filename(text):
  13         return sha1(text).hexdigest()[:16] + ".svg"
  14 
  15 def macro_QR(macro, text=None):
  16         if text is None:
  17                 text = macro.request.getQualifiedURL(macro.request.page.url(macro.request))
  18 
  19         prefix, suffix = svg_path(macro.request, macro.request.page.page_name, text)
  20         if not os.path.exists(prefix):
  21                 os.mkdir(prefix)
  22 
  23         filename = os.path.join(prefix, suffix)
  24         if not os.path.exists(filename):
  25                 p = Popen(["qrencode", "-o", filename] + qr_options, stdin=PIPE)
  26                 p.stdin.write(text)
  27                 p.stdin.close()
  28                 p.wait()
  29                 os.chmod(filename, 0664)
  30 
  31         img_url = svg_url(macro.request, macro.request.page.page_name, text)
  32         return macro.formatter.image(img_url, alt=text, title=text)

Adjust permissions to directories

   1 chmod o+rx /srv/moin/rs_admin/data{,/qr}

Configure Apache2-Directory like above #Apache2 Config and restart apache2

   1 systemctl restart apache2.service

Can be used by <<QR>>, which encodes current-page URL or
with a specifed Text like <<QR(TEXT_to/be.ENcoded)>>.

Example QRCode.

LaTeX

All Credits and Thanks to: Johannes Berg <johannes@sipsolutions.net>
https://johannes.sipsolutions.net/Projects/new-moinmoin-latex/

   1 aptitude install texlive dvipng texlive-fonts-extra

Create the parser /srv/moin/rs_admin/data/plugin/parser/latex.py

   1 """
   2 New latex formatter using dvipng and tempfile
   3 Author: JohannesBerg <johannes@sipsolutions.net>
   4 This parser (and the corresponding macro) was tested with Python 2.3.4 and
   5  * Debian Linux with out-of-the-box tetex-bin and dvipng packages installed
   6  * Windows XP (not by me)
   7 
   8 In the parser, you can add stuff to the prologue by writing
   9 %%end-prologue%%
  10 somewhere in the document, before that write stuff like \\usepackage and after it put
  11 the actual latex display code.
  12 """
  13 Dependencies = []
  14 import sha, os, tempfile, shutil, re
  15 from MoinMoin.action import AttachFile
  16 from MoinMoin.Page import Page
  17 latex_template = r'''
  18 \documentclass[12pt]{article}
  19 \pagestyle{empty}
  20 \usepackage[utf8]{inputenc}
  21 %(prologue)s
  22 \begin{document}
  23 %(raw)s
  24 \end{document}
  25 '''
  26 max_pages = 10
  27 MAX_RUN_TIME = 5 # seconds
  28 latex = "latex"    # edit full path here, e.g. reslimit = "C:\\path\\to\\latex.exe"
  29 dvipng = "dvipng"  # edit full path here (or reslimit = r'C:\path\to\latex.exe')
  30 # last arg must have %s in it!
  31 latex_args = ("--interaction=nonstopmode", "%s.tex")
  32 # last arg must have %s in it!
  33 dvipng_args = ("-bgTransparent", "-Ttight", "--noghostscript", "-l%s" % max_pages, "%s.dvi")
  34 # this is formatted with hexdigest(texcode),
  35 # page number and extension are appended by
  36 # the tools
  37 latex_name_template = "latex_%s_p"
  38 # keep this up-to-date, also with max_pages!!
  39 latex_attachment = re.compile((latex_name_template+'%s%s') % (r'[0-9a-fA-F]{40}', r'[0-9]{1,2}', r'\.png'))
  40 anchor = re.compile(r'^%%anchor:[ ]*([a-zA-Z0-9_-]+)$', re.MULTILINE | re.IGNORECASE)
  41 # the anchor re must start with a % sign to be ignored by latex as a comment!
  42 end_prologue = '%%end-prologue%%'
  43 def call_command_in_dir_NT(app, args, targetdir):
  44     reslimit = "runlimit.exe" # edit full path here
  45     os.environ['openin_any'] = 'p'
  46     os.environ['openout_any'] = 'p'
  47     os.environ['shell_escape'] = 'f'
  48     stdouterr = os.popen('%s %d "%s" %s %s < NUL' % (reslimit, MAX_RUN_TIME, targetdir, app, ' '.join(args)), 'r')
  49     output = ''.join(stdouterr.readlines())
  50     err = stdouterr.close()
  51     if not err is None:
  52         return ' error! exitcode was %d, transscript follows:\n\n%s' % (err,output)
  53     return None
  54 def call_command_in_dir_unix(app, args, targetdir):
  55     # this is the unix implementation
  56     (r,w) = os.pipe()
  57     pid = os.fork()
  58     if pid == -1:
  59       return 'could not fork'
  60     if pid == 0:
  61       try:
  62         # child
  63         os.close(r)
  64         os.dup2(os.open("/dev/null", os.O_WRONLY), 0)
  65         os.dup2(w, 1)
  66         os.dup2(w, 2)
  67         os.chdir(targetdir)
  68         os.environ['openin_any'] = 'p'
  69         os.environ['openout_any'] = 'p'
  70         os.environ['shell_escape'] = 'f'
  71         try:
  72           import resource
  73           resource.setrlimit(resource.RLIMIT_CPU,
  74                              (MAX_RUN_TIME * 1000, MAX_RUN_TIME * 1000)) # docs say this is seconds, but it is msecs on my system.
  75         except ValueError:
  76           pass
  77         # os.execvp will raise an exception if the executable isn't
  78         # present. [[ try os.execvp("aoeu", ['aoeu'])  ]]
  79         # If we don't catch exceptions here, it will be caught at the
  80         # main body below, and then os.rmdir(tmpdir) will be called
  81         # twice, once for each fork.  The second one raises an exception
  82         # in the main code, which gets back to the user.  This is bad.
  83         try:
  84           os.execvp(app, [app] + list(args))
  85         finally:
  86           print "failed to exec()",app
  87           os._exit(2)
  88       finally:
  89         os._exit(3)
  90     else:
  91       # parent
  92       os.close(w)
  93       r = os.fdopen(r,"r")
  94       output = ''.join(r.readlines())
  95       (npid, exi) = os.waitpid(pid, 0)
  96       r.close()
  97       sig = exi & 0xFF
  98       stat = exi >> 8
  99       if stat != 0 or sig != 0:
 100         return ' error! exitcode was %d (signal %d), transscript follows:\n\n%s' % (stat,sig,output)
 101       return None
 102     # notreached
 103 if os.name == 'nt':
 104     call_command_in_dir = call_command_in_dir_NT
 105 else:
 106     call_command_in_dir = call_command_in_dir_unix
 107 class Parser:
 108     extensions = ['.tex']
 109     def __init__ (self, raw, request, **kw):
 110         self.raw = raw
 111         if len(self.raw)>0 and self.raw[0] == '#':
 112             self.raw[0] = '%'
 113         self.request = request
 114         self.exclude = []
 115         if not hasattr(request, "latex_cleanup_done"):
 116             request.latex_cleanup_done = {}
 117 
 118     def cleanup(self, pagename):
 119         attachdir = AttachFile.getAttachDir(self.request, pagename, create=1)
 120         for f in os.listdir(attachdir):
 121             if not latex_attachment.match(f) is None:
 122                 os.remove("%s/%s" % (attachdir, f))
 123     def _internal_format(self, formatter, text):
 124         tmp = text.split(end_prologue, 1)
 125         if len(tmp) == 2:
 126             prologue,tex=tmp
 127         else:
 128             prologue = ''
 129             tex = tmp[0]
 130         if callable(getattr(formatter, 'johill_sidecall_emit_latex', None)):
 131           return formatter.johill_sidecall_emit_latex(tex)
 132         return self.get(formatter, tex, prologue, True)
 133 
 134     def format(self, formatter):
 135         self.request.write(self._internal_format(formatter, self.raw))
 136 
 137     def get(self, formatter, inputtex, prologue, para=False):
 138         if not self.request.latex_cleanup_done.has_key(self.request.page.page_name):
 139             self.request.latex_cleanup_done[self.request.page.page_name] = True
 140             self.cleanup(self.request.page.page_name)
 141         if len(inputtex) == 0: return ''
 142         if callable(getattr(formatter, 'johill_sidecall_emit_latex', None)):
 143           return formatter.johill_sidecall_emit_latex(inputtex)
 144         extra_preamble = ''
 145         preamble_page = self.request.pragma.get('latex_preamble', None)
 146         if preamble_page is not None:
 147           extra_preamble = Page(self.request, preamble_page).get_raw_body()
 148         extra_preamble = re.sub(re.compile('^#'), '%', extra_preamble)
 149         tex = latex_template % { 'raw': inputtex, 'prologue': extra_preamble + prologue }
 150         enctex = tex.encode('utf-8')
 151         fn = latex_name_template % sha.new(enctex).hexdigest()
 152 
 153         attachdir = AttachFile.getAttachDir(self.request, formatter.page.page_name, create=1)
 154         dst = "%s/%s%%d.png" % (attachdir, fn)
 155         if not os.access(dst % 1, os.R_OK):
 156           tmpdir = tempfile.mkdtemp()
 157           try:
 158               data = open("%s/%s.tex" % (tmpdir, fn), "w")
 159               data.write(enctex)
 160               data.close()
 161               args = list(latex_args)
 162               args[-1] = args[-1] % fn
 163               res = call_command_in_dir(latex, args, tmpdir)
 164               if not res is None:
 165                   return formatter.preformatted(1)+formatter.text('latex'+res)+formatter.preformatted(0)
 166               args = list(dvipng_args)
 167               args[-1] = args[-1] % fn
 168               res = call_command_in_dir(dvipng, args, tmpdir)
 169               if not res is None:
 170                   return formatter.preformatted(1)+formatter.text('dvipng'+res)+formatter.preformatted(0)
 171               page = 1
 172               while os.access("%s/%s%d.png" % (tmpdir, fn, page), os.R_OK):
 173                   shutil.copyfile ("%s/%s%d.png" % (tmpdir, fn, page), dst % page)
 174                   page += 1
 175 
 176           finally:
 177               for root,dirs,files in os.walk(tmpdir, topdown=False):
 178                   for name in files:
 179                       os.remove(os.path.join(root,name))
 180                   for name in dirs:
 181                       os.rmdir(os.path.join(root,name))
 182               os.rmdir(tmpdir)
 183         result = ""
 184         page = 1
 185         loop = False
 186         for match in anchor.finditer(inputtex):
 187             result += formatter.anchordef(match.group(1))
 188         for match in anchor.finditer(prologue):
 189             result += formatter.anchordef(match.group(1))
 190         while os.access(dst % page, os.R_OK):
 191             url = AttachFile.getAttachUrl(formatter.page.page_name, fn+"%d.png" % page, self.request)
 192             if loop:
 193                 result += formatter.linebreak(0)+formatter.linebreak(0)
 194             if para:
 195                 result += formatter.paragraph(1)
 196             result += formatter.image(src="%s" % url, alt=inputtex, title=inputtex, align="absmiddle")
 197             if para:
 198                 result += formatter.paragraph(0)
 199             page += 1
 200             loop = True
 201         return result

Create the parser /srv/moin/rs_admin/data/plugin/macro/latex.py

   1 """
   2 See the latex parser, this is just a thin wrapper around it.
   3 """
   4 # Imports
   5 from MoinMoin import wikiutil
   6 import re
   7 Dependencies = []
   8 splitre = re.compile(r'([^\\])%')
   9 class latex:
  10     def __init__(self, macro, args):
  11         self.macro = macro
  12         self.formatter = macro.formatter
  13         self.text = args
  14     
  15     def renderInPage(self):
  16         # return immediately if getting links for the current page
  17         if self.macro.request.mode_getpagelinks:
  18             return ''
  19         if self.text is None: # macro call without parameters
  20             return ''
  21         # get an exception? for moin before 1.3.2 use the following line instead:
  22         # L = wikiutil.importPlugin('parser', 'latex', 'Parser', self.macro.cfg.data_dir)
  23         L = wikiutil.importPlugin(self.macro.cfg, 'parser', 'latex', 'Parser')
  24         if L is None:
  25             return self.formatter.text("<<please install the latex parser>>")
  26         l = L('', self.macro.request)
  27         tmp = splitre.split(self.text, 1)
  28         if len(tmp) == 3:
  29             prologue,p2,tex=tmp
  30             prologue += p2
  31         else:
  32             prologue = ''
  33             tex = tmp[0]
  34         return l.get(self.formatter, tex, prologue)
  35 def execute(macro, args):
  36     return latex(macro, args).renderInPage()

Fix Permissions for plugins and restart apache2

   1 systemctl restart apache2.service

Works: Latex

Asciinema

Macro Call:

<<Asciinema(filename=None, pagename=None, theme=None, cols=None, rows=24, autoplay=None, preload=None, loop=None, start_at=None, speed=1, poster="npt:1", font_size="small", title="asciinema", author=None, author_url=None, author_img_url=None)>>

Prepare static files

   1 install -o root -g root -m 755 /usr/share/moin/htdocs/asciinema
   2 cd /usr/share/moin/htdocs/asciinema
   3 wget https://github.com/asciinema/asciinema-player/releases/download/v2.6.1/asciinema-player.js
   4 wget https://github.com/asciinema/asciinema-player/releases/download/v2.6.1/asciinema-player.css
   5 chmod 644 /usr/share/moin/htdocs/asciinema/*

Source: /srv/moin/rs_admin/data/plugin/macro/Asciinema.py

   1 # -*- coding: utf-8 -*-
   2 """
   3     MoinMoin - Asciinema
   4 
   5     This macro allows to attach asciinema cast-files to a page and
   6     play them. Maybe you like it, too.
   7     
   8     Based on and inspired by:
   9     https://github.com/asciinema/asciinema-player
  10 
  11     Thanks for this cool set of tools to the asciinema-project!
  12     Thanks for MoinMoin!
  13 
  14     The github page also documents all the asciinema-player options.
  15     (I just snake_cased the options for the plugin).
  16 
  17     Installation:
  18     1. Drop the latest release (.js+.css) of the following github-repo
  19        into your static files (Probably into:
  20        "/usr/share/moin/htdocs/asciinema")
  21     2. Drop the macro in your site-specific "data/plugin/macro" directory.
  22     3. Restart server.
  23     4. Attach .cast file to site.
  24     5. Use macro.
  25     
  26     
  27     Usage:
  28     <<Asciinema(attached_file.cast, pagename, theme, cols, rows
  29         autoplay, preload, loop, start_at,
  30         speed, poster, font_size,
  31         title, author, author_url, author_img_url)>>
  32 
  33     Some hints:
  34     cols=([80]|Num)
  35     rows=([24]|Num)
  36     
  37     font-size=([small]|medium|big|Num(px,em,…)
  38     
  39     poster=(npt:mm:ss|"data:text/plain,Poster text")
  40     
  41     theme=([asciinema]|tango|solarized-dark|solarized-light|monokai)
  42 
  43     @copyright: 2019 Tobias Stein
  44     @license: GNU GPLv2 or later
  45 
  46     I'm not a frontend-, web- or python-programmer,
  47     i just wanted to have sth. like this.
  48     If you like - your free to improve code quality.
  49 
  50     No warranty for nothing, usage on your on own responsibility.
  51 
  52     Changelog:
  53     * 1.0:
  54       * Working implementation
  55       * Probably vulnerable to javescript injection
  56         "author_url" and "author_img_url"
  57 """
  58 
  59 import re
  60 from MoinMoin import wikiutil
  61 from MoinMoin.action.AttachFile import getAttachUrl, getFilename, exists
  62 from MoinMoin.Page import Page
  63 
  64 generates_headings = False
  65 
  66 def macro_Asciinema(
  67         macro, filename=None, pagename=None,
  68         theme=None, cols=None, rows=24,
  69         autoplay=None, preload=None, loop=None, start_at=None,
  70         speed="1.0", poster="npt:1", font_size="small",
  71         title="Asciinema", author=None, author_url=None, author_img_url=None):
  72 
  73     if not pagename:
  74         pagename = macro.formatter.page.page_name
  75 
  76     request = macro.request
  77     formatter = macro.formatter
  78 
  79     if not exists(request, pagename, filename):
  80         return formatter.text(
  81                 'Attachment "%s" does not exist on page "%s"'
  82                 % (filename, pagename))
  83     else:
  84 
  85         src=getAttachUrl(pagename, filename, request)
  86         player_opts = []
  87         if theme is not None:
  88                 pattern = re.compile("^(asciinema|tango|solarized-dark|solarized-light|monokai)$")
  89                 if pattern.match(theme):
  90                         player_opts.extend([formatter.rawHTML('theme="%s"' % theme)])
  91                 else:
  92                         return formatter.text(
  93                                 'Theme is unknown.')
  94         if rows is not None:
  95                 if isinstance(rows, int):
  96                         player_opts.extend([formatter.rawHTML('rows="%s"' % rows)])
  97                 else:
  98                     return formatter.text(
  99                         'Unable to determine rows.')
 100         if cols is not None:
 101                 if isinstance(cols, int):
 102                         player_opts.extend([formatter.rawHTML('cols="%s"' % cols)])
 103                 else:
 104                     return formatter.text(
 105                         'Unable to determine cols.')
 106         if autoplay is not None:
 107                 player_opts.extend([formatter.rawHTML('autoplay="%s"' % True)])
 108         if preload is not None:
 109                 player_opts.extend([formatter.rawHTML('preload="%s"' % True)])
 110         if loop is not None:
 111                 player_opts.extend([formatter.rawHTML('loop="%s"' % True)])
 112         if start_at is not None:
 113                 player_opts.extend([formatter.rawHTML('start-at="%s"' % start_at)])
 114         if speed is not None:
 115                 if isinstance(speed, (int, float)):
 116                         player_opts.extend([formatter.rawHTML('speed="%s"' % speed)])
 117                 elif isinstance(float(speed), (int, float)):
 118                         player_opts.extend([formatter.rawHTML('speed="%s"' % float(speed))])
 119                 else:
 120                     return formatter.text(
 121                         'Unable to determine speed.')
 122         if poster is not None:
 123                 player_opts.extend([formatter.rawHTML('poster="%s"' % poster)])
 124         if font_size is not None:
 125                 pattern = re.compile("^(small|medium|big|\d\+(px|r?em))$")
 126                 if pattern.match(font_size):
 127                     player_opts.extend([formatter.rawHTML('font-size="%s"' % font_size)])
 128                 else:
 129                     return formatter.text(
 130                         'Unable to determine font-size.')
 131         if title is not None:
 132                 title_html = wikiutil.escape(title)
 133                 title_html = title_html.strip()
 134                 player_opts.extend([formatter.rawHTML('title="%s"' % title_html)])
 135         if author is not None:
 136                 author_html = wikiutil.escape(author)
 137                 author_html = author_html.strip()
 138                 player_opts.extend([formatter.rawHTML('author="%s"' % author_html)])
 139         if author_url is not None:
 140                 player_opts.extend([formatter.rawHTML('author-url="%s"' % author_url)])
 141         if author_img_url is not None:
 142                 player_opts.extend([formatter.rawHTML('author-img-url="%s"' % author_img_url)])
 143 
 144         ret = []
 145         ret.extend([formatter.rawHTML(asciinemaRessource(request, "asciinema-player", "css"))])
 146         ret.extend([formatter.rawHTML(asciinemaRessource(request, "asciinema-player", "js"))])
 147         ret.extend([formatter.rawHTML(
 148                 '<asciinema-player %s src="%s"></asciinema-player>' %
 149                 (' '.join(player_opts), src))])
 150 
 151         return ''.join(ret)
 152 
 153 
 154 def asciinemaRessource(request, name, suffix):
 155     """ Format external script html """
 156     uri = '%s/asciinema/%s.%s' % (request.cfg.url_prefix_static, name, suffix)
 157     if suffix == 'css':
 158         tag = '<link rel="stylesheet" type="text/css" media="all" href="%s">' % uri
 159     if suffix == 'js':
 160         tag = '<script type="text/javascript" src="%s"></script>' % uri
 161     return tag

YouTube

Display a single YouTube flash video as emended or display some related videos (pagename or keyword) in a videobar where a user can view them.

MoinMo.in MacroMarket Youtube

   1 cd /srv/moin/rs_admin/data/plugin/macro
   2 wget 'http://moinmo.in/MacroMarket/YouTube?action=AttachFile&do=get&target=YouTube-0.6.py' \
   3         -O YouTube.py
   4 systemctl restart apache2

Well actually the version from the MacroMarket, was based on Flash, which my browser does not support anymore. So i rewrote this plugin to use iframes. I'll bring it upstream, soon.

In the meantime here you are: /srv/moin/rs_admin/data/plugin/macro/YouTube.py

   1 # -*- coding: iso-8859-1 -*-
   2 u"""
   3     MoinMoin - YouTube macro Version 0.6
   4                Displays an embedded object with the wanted video or
   5                displays a list of videos (videobar).
   6 
   7     <<YouTube(video_id="YouTubeID", keyword="keyword")>>
   8     Default is to use the current pagename as keyword.
   9 
  10     Examples:
  11      * <<YouTube>>
  12      * <<YouTube(video_id=2SXKM-dLJV8)>>
  13      * <<YouTube(keyword="iron maiden")>>
  14 
  15     @copyright: 2008 by MarcelH�fner (http://moinmo.in/MarcelH�fner),
  16                 2010 by ThomasWaldmann (http://moinmo.in/ThomasWaldmann)
  17     @license: GNU GPL, see COPYING for details.
  18 
  19     Attention! The "keyword" part is created with the "Google AJAX Search API Tools".
  20     Licence Information can be viewed under: http://code.google.com/intl/de-DE/apis/ajaxsearch/terms.html
  21 
  22     @TODO:
  23      * Change the hardcoded https string into http/https depending on the requested url.
  24      * Don't use the google ajax stuff.
  25      * Check if youtube videos are disabled (mimetypes_xss).
  26 
  27      https://developers.google.com/youtube/player_parameters
  28 
  29 """
  30 
  31 from MoinMoin import wikiutil
  32 
  33 def macro_YouTube(macro, video_id, playlist_id, user_id, query, start_at, width=314, height=176, frameborder=0):
  34     formatter = macro.formatter
  35     request = macro.request
  36     _ = request.getText
  37     html = u""
  38     parameters = {}
  39     ret = []
  40     usage = wikiutil.escape(
  41             '<<YouTube(((video|playlist|user)_id=yu0tUb3ID|query=QUERY_STRING)[, width=int][, height=int][, start_at=int][, frameborder=int])>>')
  42 
  43     if video_id is None and playlist_id is None and user_id is None and query is None:
  44         # When no video_id or playlist_id is given, show usage.
  45         html = u"""
  46                 <p>
  47                     Unable to determine Youtube-Video to be embedded.<br>
  48                     Usage:
  49                     <pre>%s</pre>
  50                 </p>
  51                 """ % usage
  52 
  53     ### PARSING OF COMMON PARAMETERS
  54     if width is not None:
  55             if isinstance(width, int):
  56                     parameters.update(
  57                             {"width": formatter.rawHTML('width="%s"' % width)})
  58             else:
  59                 return formatter.text(
  60                     'Unable to determine width. %s' % width)
  61 
  62     if height is not None:
  63             if isinstance(height, int):
  64                     parameters.update(
  65                             {"height": formatter.rawHTML('height="%s"' % height)})
  66             else:
  67                 return formatter.text(
  68                     'Unable to determine height.')
  69 
  70     if frameborder is not None:
  71             if isinstance(frameborder, int):
  72                     parameters.update(
  73                             {"frameborder": formatter.rawHTML('frameborder="%s"' % frameborder)})
  74             else:
  75                 return formatter.text(
  76                     'Unable to determine frameborder.')
  77 
  78 
  79     ### EMBED A PLAYLIST BY ITS UPLOAD USER VIA IFRAME
  80     if user_id is not None:
  81         url_prefix = u"https://www.youtube.com/embed?listType=user_uploads&list="
  82         parameters.update({"user_id":  wikiutil.escape(user_id)})
  83         parameters.update(
  84                 {"src": formatter.rawHTML('src="%s%s"' %
  85                     (url_prefix, parameters['user_id']))})
  86 
  87         html = u"""
  88         <iframe
  89             %(src)s
  90             %(width)s %(height)s %(frameborder)s
  91             allow="autoplay; encrypted-media" allowfullscreen>
  92         </iframe>
  93         """ % parameters
  94 
  95     ### EMBED A PLAYLIST BY A SEARCH QUERY VIA IFRAME
  96     if query is not None:
  97         url_prefix = u"http://www.youtube.com/embed?listType=search&list="
  98         parameters.update({"query":  wikiutil.escape(query)})
  99         parameters.update(
 100                 {"src": formatter.rawHTML('src="%s%s"' %
 101                     (url_prefix, parameters['query']))})
 102 
 103         html = u"""
 104         <iframe
 105             %(src)s
 106             %(width)s %(height)s %(frameborder)s
 107             allow="autoplay; encrypted-media" allowfullscreen>
 108         </iframe>
 109         """ % parameters
 110 
 111     ### EMBED A PLAYLIST BY ITS PLAYLIST_ID VIA IFRAME
 112     if playlist_id is not None:
 113         url_prefix = u"https://www.youtube.com/embed/videoseries?list="
 114         parameters.update({"playlist_id":  wikiutil.escape(playlist_id)})
 115         parameters.update(
 116                 {"src": formatter.rawHTML('src="%s%s"' %
 117                     (url_prefix, parameters['playlist_id']))})
 118 
 119         html = u"""
 120         <iframe
 121             %(src)s
 122             %(width)s %(height)s %(frameborder)s
 123             allow="autoplay; encrypted-media" allowfullscreen>
 124         </iframe>
 125         """ % parameters
 126 
 127     ### EMBED A SINGLE VIDEO BY ITS VIDEO_ID VIA IFRAME
 128     if video_id is not None:
 129         url_prefix = u"https://www.youtube-nocookie.com/embed/"
 130         parameters.update({"video_id":  wikiutil.escape(video_id)})
 131 
 132         ### PARSING OF VIDEO PARAMETERS
 133         if start_at is not None:
 134                 if isinstance(start_at, int):
 135                         #iframe_opts.extend([formatter.rawHTML('?start="%s"' % start_at)])
 136                         parameters.update(
 137                                 {"src": formatter.rawHTML(
 138                                     'src="%s%s?start=%s"' %
 139                                     (url_prefix, video_id, start_at))})
 140                 elif isinstance(int(start_at), int):
 141                         #iframe_opts.extend([formatter.rawHTML('?start="%s"' % int(start_at))])
 142                         parameters.update(
 143                                 {"src": formatter.rawHTML(
 144                                     'src="%s%s?start=%s"' %
 145                                     (url_prefix, parameters['video_id'], int(start_at)))})
 146                 else:
 147                     return formatter.text(
 148                         'Unable to determine starting position.')
 149         else:
 150                 parameters.update(
 151                         {"src": formatter.rawHTML('src="%s%s"' %
 152                             (url_prefix, video_id))})
 153 
 154         ## EMBED VIDEO IFRAME IN ADVANCED PRIVACY MODE
 155         html = u"""
 156         <iframe 
 157             %(src)s
 158             %(width)s %(height)s %(frameborder)s
 159             allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen>
 160         </iframe>
 161         """ % parameters
 162 
 163     ret.extend([formatter.rawHTML(html)])
 164         
 165     return ''.join(ret)

Helper Scripts

Check ACLs from CLI

   1 for DIRECTORY in /usr/share/moin/rs_admin/data/pages/*; do
   2         if [ -e $DIRECTORY/current ]; then
   3                 CURRENT="$(cat $DIRECTORY/current)"
   4                 if [ -e "$DIRECTORY/revisions/$CURRENT" ]; then
   5                         grep -H '^\s*#acl' "$DIRECTORY/revisions/$CURRENT" 2>&1
   6                 fi
   7         fi
   8 done \
   9         |sed -r 's/(.+):(#acl .+)$/\1\t\2/'\
  10         |column -t

Align file encoding

Your file might already be in the right encoding.

Remember the first 256 chars of ISO-8859-1 are the same.
Remember the first 127 chars (lower 7-bit) are the same in ASCII, ISO-8859-1 and UTF-8.

encoding_align.sh

   1 #!/bin/bash
   2 
   3 DRYRUN=false
   4 INTERACTIVE=false
   5 VERBOSE=false
   6 PROGRAM="${0%%*/}"
   7 
   8 usage () {
   9         cat <<-EOF
  10                 This script recursivly changes file encodings of
  11                 python scripts (*.py) to a target encoding
  12                 stored in a comment like '# -*- coding: … -*-' or
  13                 was explicitly specified.
  14 
  15                 $PROGRAM OPTIONS -- /path/to/file [/path/to/diretory]
  16 
  17                 Where options are
  18                         [-d|--dry-run]          Do not change anything.
  19                         [-i|--interactive]      Prompt for confirmation.
  20                         [-f|--force-encoding]   Override output encoding.
  21                         [-h|--help]             Display this page.
  22                         [-v|--verbose]          Show additional information.
  23                                                 (Implies verbosity).
  24 
  25         As always this programm comes as is and without any warranty.
  26         Copyright:      Rockstable IT.
  27         License:        GPLv2
  28 
  29         EOF
  30 
  31         exit 0
  32 }
  33 
  34 
  35 
  36 unset ENC_FORCE
  37 
  38 # Note that we use "$@" to let each command-line parameter expand to a
  39 # separate word. The quotes around "$@" are essential!
  40 # We need TEMP as the 'eval set --' would nuke the return value of getopt.
  41 TEMP=$(getopt -o 'dif:hv' \
  42                 --long 'dry-run,interactive,help,force-encoding:,verbose' \
  43                 -n "$PROGRAM" -- "$@")
  44 
  45 if [ $? -ne 0 ]; then
  46         echo 'Terminating...' >&2
  47         exit 1
  48 fi
  49 
  50 # Note the quotes around "$TEMP": they are essential!
  51 eval set -- "$TEMP"
  52 unset TEMP
  53 
  54 while true; do
  55         case "$1" in
  56                 '-d'|'--dry-run')
  57                         DRYRUN=true
  58                         shift
  59                         continue
  60                 ;;
  61                 '-i'|'--interactive')
  62                         VERBOSE=true
  63                         INTERACTIVE=true
  64                         shift
  65                         continue
  66                 ;;
  67                 '-h'|'--help')
  68                         usage
  69                         shift
  70                         continue
  71                 ;;
  72                 '-v'|'--verbose')
  73                         VERBOSE=true
  74                         shift
  75                         continue
  76                 ;;
  77                 '-f'|'--force-encoding')
  78                         ENC_FORCE="$2"
  79                         shift 2
  80                         continue
  81                 ;;
  82                 '--')
  83                         shift
  84                         break
  85                 ;;
  86                 *)
  87                         echo 'Internal error!' >&2
  88                         exit 1
  89                 ;;
  90         esac
  91 done
  92 
  93 ### SANITY CHECKS
  94 if [ "${#@}" -ne 0 ]; then
  95         for PATH_TARGET in "$@"; do
  96                 if ! [ -d "$PATH_TARGET" ] \
  97                 && ! [ -f "$PATH_TARGET" ];then
  98                         echo "Target '$PATH_TARGET' is neither a file nor a directory. Exiting…"
  99                         exit 1
 100                 fi
 101         done
 102 else
 103         echo "Target path is missing. Exiting…"
 104         exit 1
 105 fi
 106 
 107 ### FUNCTIONS
 108 encoding_check () {
 109         local ENC="$1"
 110         if [ -z "$ENC" ]; then
 111                 $VERBOSE && cat <<-EOF
 112                         Encoding '$ENC' is missing. Skipping.
 113                 EOF
 114                 return 1
 115         elif ! iconv -l |grep -qi '^'"$ENC"'//'; then
 116                 $VERBOSE && cat <<-EOF
 117                         Encoding '$ENC' is not supported by iconv. Skipping…"
 118                 EOF
 119                 return 1
 120         fi
 121 }
 122 
 123 dryrun_or_execute () {
 124         if $DRYRUN;then
 125                 eval echo "Dryrun: $COMMAND"
 126         else
 127                 eval $COMMAND
 128         fi
 129 }
 130 
 131 
 132 encoding_align () {
 133         local CURRENT="$1"
 134         local TARGET="$2"
 135         local FILE="$3"
 136         local REPLY
 137 
 138         if [ -n "$CURRENT" ] && [ -n "$TARGET" ] && [ -n "$FILE" ]; then
 139                 COMMAND='iconv -f "$CURRENT" -t "$TARGET" -o "$FILE" "$FILE"'
 140                 if $INTERACTIVE;then
 141                         read -r -N 1 -p "Change encoding of file '$FILE'? [yN] " REPLY
 142                         echo
 143                         if grep -qi y <<< "$REPLY";then
 144                                 dryrun_or_execute
 145                         else
 146                                 echo "Skipping…"
 147                         fi
 148                 else
 149                         dryrun_or_execute
 150                 fi
 151         else
 152                 echo "Either current or target encoding or file name is missing. Skipping…" 
 153         fi
 154 }
 155 
 156 to_upper () {
 157         local INPUT="$1"
 158         tr 'a-z' 'A-Z' <<< "$INPUT"
 159 }
 160 
 161 ### MAIN
 162 FILES="$(find "$@" -type f -name '*.py')"
 163 
 164 for FILE in $FILES; do
 165         ### UNCERTAIN (e.g. "ASCII", "ISO-8859")
 166         #ENC_CURRENT="$(file -bi "$FILE" \
 167         #       |cut -d\: -f2 \
 168         #       |sed -r 's/ .*?, ([[:alnum:]-]+) text( executable)?(, .+)?/\1/')"
 169         ### MORE CERTAIN "us-ascii", "iso-8859-1"
 170         ENC_CURRENT_RAW="$(file -bi "$FILE" \
 171                 |cut -d\; -f2 \
 172                 |sed -r 's/ charset=([[:alnum:]-]+)/\1/')"
 173         echo $ENC_CURRENT_RAW|to_upper
 174         ENC_CURRENT="$(to_upper "$ENC_CURRENT_RAW";)"
 175         ENC_TARGET_RAW="$(grep -E -- '-\*- coding:' $FILE \
 176                 |sed -r 's/# -\*- coding: ([[:alnum:]-]+) .*$/\1/')"
 177         ENC_TARGET="$(to_upper "$ENC_TARGET_RAW";)"
 178 
 179         if [ -n "$ENC_FORCE" ]; then
 180                ENC_TARGET="$ENC_FORCE"
 181                echo "Forcing target encoding of '$ENC_TARGET'"
 182         fi
 183 
 184         $VERBOSE && cat <<-EOF
 185 
 186                 File: '$FILE'
 187                 Current encoding: '$ENC_CURRENT'
 188                 Target encoding: '$ENC_TARGET'
 189         EOF
 190 
 191         if [ "$ENC_CURRENT" != "$ENC_TARGET" ]; then
 192                 $VERBOSE && echo "Encoding of file '$FILE' needs to be aligned."
 193 
 194                 encoding_check "$ENC_CURRENT" || continue
 195                 encoding_check "$ENC_TARGET" || continue
 196                 #read REPLY
 197                 #echo $REPLY
 198 
 199                 encoding_align "$ENC_CURRENT" "$ENC_TARGET" "$FILE"
 200         fi
 201 done

MoinMoin

About

Links

Install

Configure

Apache2 Config

Farmconfig

Moin Instances

Decisions

Wikiconfig

Create groups and users

Instance Directories

Permissions

Trouble-Shooting

Spammers and Griefers

TextChas

Disable Account Creation

Free wiki from bot accounts

Search engine optimization - SEO

Maintainance

cleancache

makecache

cleansessions

cleanpage

Customization

Theme Memodump

MOBILE FIRST CSS Adjustments

Minify css

Uglify js

Calculate Sub-Resource Integrity Hash

Xapian Search/Index

Xapian Search/Index - create and enable

Xapian Search/Index - rebuild

QR-Codes

LaTeX

Asciinema

YouTube

Helper Scripts

Check ACLs from CLI

Align file encoding