Difference between revisions of "Mwlib"

From SME Server
Jump to navigationJump to search
 
(15 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
{{WIP box}}
 
{{WIP box}}
  
−
this how-to is an help to install mediawiki libraries (mwlib) on SME 9
+
this how-to is an help to install mediawiki libraries (mwlib) on SME 9 and SME 10
  
 
=== install ===
 
=== install ===
 +
<tabs container=""><tab name="For SME 10">
 +
<syntaxhighlight lang="bash">
 +
yum install smeserver-extrarepositories-epel -y
 +
 +
yum install pdftk gcc gcc-c++ cpp make python-devel libjpeg-devel zlib-devel freetype-devel  lcms-devel lcms2-devel libxslt-devel libxml2-devel git ocaml python-pip python-gevent python-py python-apipkg  ImageMagick texlive-latex texlive-texvc blahtexml openssl-devel python-devel libffi-devel  pyOpenSSL python2-pyasn1 python2-pyasn1-modules--enablerepo=epel --enablerepo=extras -y
 +
</syntaxhighlight>
 +
note, those are not available on SME10, but it seems to work out: texlive-texvc, blahtexml
 +
 +
===== python pip =====
 +
<syntaxhighlight lang="bash">
 +
# apipkg provided in the repo is 1.4 and does not work with python 2.6
 +
pip install  apipkg==1.2
 +
# Pillow is required, but the version 4.0 has a syntax error when running with python 2.6
 +
pip install -i http://pypi.pediapress.com/simple/ Pillow==3.4.2 --trusted-host pypi.pediapress.com
 +
# last version fails with python 2.6
 +
pip install -i http://pypi.pediapress.com/simple/ --trusted-host pypi.pediapress.com  pygments==1.6
 +
easy_install -i http://pypi.pediapress.com/simple/  mwlib mwlib.rl  pyfribidi
 +
cd ~
 +
git clone https://github.com/pediapress/texvc
 +
cd texvc; make; make install
 +
 +
pip install ndg-httpsclient
 +
 +
</syntaxhighlight>
 +
 +
===== configure services =====
 +
<syntaxhighlight lang="bash">
 +
mkdir /var/log/mediawiki/
 +
chown www:www -R /var/log/mediawiki/
 +
config set nserve service status enabled access localhost TCPPort 19737
 +
config set mw-qserve service status enabled access localhost TCPPort 19252
 +
config set nslave service status enabled access localhost
 +
 +
cat  >/etc/systemd/system/nslave.service <<- EOM
 +
[Unit]
 +
Description=Mediawiki_collection_extension nslave
 +
After=network-online.target mw-qserve.service  nserve.service
 +
 +
[Service]
 +
Type=simple
 +
User=www
 +
Group=www
 +
Environment=PATH=/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin
 +
ExecStart=/bin/bash -c 'nslave --serve-files-port=20361 --host=localhost --port=19252 --numprocs=1 --cachedir /var/log/mediawiki/ '
 +
 +
[Install]
 +
WantedBy=sme-server.target
 +
 +
EOM
 +
 +
cat  >/etc/systemd/system/mw-qserve.service <<- EOM
 +
[Unit]
 +
Description=Mediawiki_collection_extension mw-qserve
 +
After=network-online.target nserve.service
 +
 +
[Service]
 +
Type=simple
 +
User=www
 +
Group=www
 +
Environment=PATH=/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin
 +
ExecStart=/bin/bash -c 'mw-qserve -p 19252 -i localhost '
 +
 +
[Install]
 +
WantedBy=sme-server.target
 +
EOM
 +
 +
 +
cat  >/etc/systemd/system/nserve.service <<- EOM
 +
[Unit]
 +
Description=Mediawiki_collection_extension nserve
 +
After=network-online.target
 +
 +
[Service]
 +
Type=simple
 +
User=www
 +
Group=www
 +
Environment=PATH=/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin
 +
ExecStart=/bin/bash -c 'nserve --port=19737 --qserve=localhost:19252 '
 +
 +
[Install]
 +
WantedBy=sme-server.target
 +
 +
EOM
 +
 +
/etc/e-smith/events/actions/systemd-default
 +
/usr/bin/systemctl start nserve.service mw-qserve.service nslave.service
 +
</syntaxhighlight>
 +
 +
 +
</tab><tab name="For SME 9">
  
 
you will need pdftk from dag (this repo does not exist anymore so here a way to get the files)
 
you will need pdftk from dag (this repo does not exist anymore so here a way to get the files)
Line 17: Line 107:
 
install needed dependencies
 
install needed dependencies
 
  yum install python-pip python-gevent python-py python-apipkg  ImageMagick texlive-latex texlive-texvc blahtexml --enablerepo=epel
 
  yum install python-pip python-gevent python-py python-apipkg  ImageMagick texlive-latex texlive-texvc blahtexml --enablerepo=epel
 +
 +
this should install<syntaxhighlight lang="bash">
 +
======================================================================================================================
 +
Installation:
 +
python-apipkg                          noarch              1.0-1.el6                    epel                  10 k
 +
python-gevent                          x86_64              0.13.8-3.el6                  epel                189 k
 +
python-pip                              noarch              7.1.0-1.el6                  epel                1.5 M
 +
python-py                              noarch              1.4.18-1.el6                  epel                307 k
 +
texlive-latex                          x86_64              2007-60.el6_7                base                  83 k
 +
Dependencies:
 +
kpathsea                                x86_64              2007-60.el6_7                base                118 k
 +
netpbm                                  x86_64              10.47.05-11.el6              base                802 k
 +
netpbm-progs                            x86_64              10.47.05-11.el6              base                1.7 M
 +
openjpeg-libs                          x86_64              1.3-16.el6_8                  updates              62 k
 +
poppler                                x86_64              0.12.4-12.el6_9              updates              559 k
 +
poppler-data                            noarch              0.4.0-1.el6                  base                2.2 M
 +
psutils                                x86_64              1.17-34.el6                  base                  46 k
 +
python-greenlet                        x86_64              0.4.2-1.el6                  epel                  24 k
 +
python-setuptools                      noarch              0.6.10-3.el6                  base                336 k
 +
tex-preview                            noarch              11.85-10.el6                  base                  50 k
 +
texlive                                x86_64              2007-60.el6_7                base                1.8 M
 +
texlive-dvips                          x86_64              2007-60.el6_7                base                193 k
 +
texlive-texmf                          noarch              2007-39.el6_7                base                2.8 M
 +
texlive-texmf-dvips                    noarch              2007-39.el6_7                base                237 k
 +
texlive-texmf-errata                    noarch              2007-7.1.el6                  base                5.3 k
 +
texlive-texmf-errata-dvips              noarch              2007-7.1.el6                  base                5.3 k
 +
texlive-texmf-errata-fonts              noarch              2007-7.1.el6                  base                5.5 k
 +
texlive-texmf-errata-latex              noarch              2007-7.1.el6                  base                5.4 k
 +
texlive-texmf-fonts                    noarch              2007-39.el6_7                base                  48 M
 +
texlive-texmf-latex                    noarch              2007-39.el6_7                base                5.3 M
 +
texlive-utils                          x86_64              2007-60.el6_7                base                254 k
 +
 +
</syntaxhighlight>
  
 
pip install :
 
pip install :
Line 27: Line 150:
 
  pip install -i http://pypi.pediapress.com/simple/ mwlib mwlib.rl  pyfribidi --trusted-host pypi.pediapress.com
 
  pip install -i http://pypi.pediapress.com/simple/ mwlib mwlib.rl  pyfribidi --trusted-host pypi.pediapress.com
  
−
install texvc
+
if the previous fails:
 +
 
 +
<syntaxhighlight  lang="bash" >
 +
easy_install -i http://pypi.pediapress.com/simple/  mwlib mwlib.rl  pyfribidi
 +
</syntaxhighlight>install texvc
 
  cd ~  
 
  cd ~  
 
  git clone https://github.com/pediapress/texvc
 
  git clone https://github.com/pediapress/texvc
Line 40: Line 167:
 
  def tostring(self): return self.tobytes()
 
  def tostring(self): return self.tobytes()
  
−
=== startup script ===
+
===== startup script =====
 
  mkdir /var/log/mediawiki/
 
  mkdir /var/log/mediawiki/
 
  chown www:www -R /var/log/mediawiki/
 
  chown www:www -R /var/log/mediawiki/
Line 79: Line 206:
 
  popd
 
  popd
  
−
logrotate :
+
===== logrotate : =====
 +
<syntaxhighlight lang="bash">
 +
cat  > /etc/logrotate.d/mwlib <<- EOM
 +
 
 +
/var/log/mediawiki/mw-serve_log.txt
 +
{
 +
        compress
 +
        weekly
 +
        rotate 4
 +
        maxsize 5M
 +
        dateext
 +
        dateformat -%Y%m%d%H%M%S
 +
        missingok
 +
        postrotate
 +
                /etc/init.d/mw-serve restart
 +
        endscript
 +
 
 +
 
 +
 
 +
}
 +
 
 +
EOM
 +
</syntaxhighlight>
 +
===You might also need to do ===
 +
if python complains too much of SSL and SNI
 +
yum install openssl-devel python-devel libffi-devel -y
 +
pip install pyOpenSSL ndg-httpsclient pyasn1
 +
 +
see https://stackoverflow.com/questions/29099404/ssl-insecureplatform-error-when-using-requests-package
 +
</tab>
 +
 
 +
</tabs>
  
 
===Sources===
 
===Sources===
Line 86: Line 244:
 
# http://edutechwiki.unige.ch/en/Mediawiki_collection_extension_installation
 
# http://edutechwiki.unige.ch/en/Mediawiki_collection_extension_installation
  
 +
 +
===Alternatives ===
 +
use rpm for centos7
 +
https://gitlab.com/harbottle/epypel
 +
 +
https://harbottle.gitlab.io/epypel/7/x86_64/
 +
 +
=== to test ===
 +
 +
mw-render --config http://en.wikipedia.org/w/ --output test.pdf --writer rl Physics
 +
or
 +
mw-zip -c https://mywiki.com/ -o test.zip "bunch of pages" "here also"
 +
mw-render -c test.zip -o test.pdf -w rl --language fr
 
----
 
----
 
[[Category:Howto]]
 
[[Category:Howto]]

Latest revision as of 03:23, 14 September 2023

Warning.png Work in Progress:
This page is a Work in Progress. The contents off this page may be in flux, please have a look at this page history the to see list of changes.


this how-to is an help to install mediawiki libraries (mwlib) on SME 9 and SME 10

install

yum install smeserver-extrarepositories-epel -y

yum install pdftk gcc gcc-c++ cpp make python-devel libjpeg-devel zlib-devel freetype-devel  lcms-devel lcms2-devel libxslt-devel libxml2-devel git ocaml python-pip python-gevent python-py python-apipkg  ImageMagick texlive-latex texlive-texvc blahtexml openssl-devel python-devel libffi-devel  pyOpenSSL python2-pyasn1 python2-pyasn1-modules--enablerepo=epel --enablerepo=extras -y

note, those are not available on SME10, but it seems to work out: texlive-texvc, blahtexml

python pip
# apipkg provided in the repo is 1.4 and does not work with python 2.6
pip install  apipkg==1.2
# Pillow is required, but the version 4.0 has a syntax error when running with python 2.6
pip install -i http://pypi.pediapress.com/simple/ Pillow==3.4.2 --trusted-host pypi.pediapress.com
# last version fails with python 2.6
pip install -i http://pypi.pediapress.com/simple/ --trusted-host pypi.pediapress.com  pygments==1.6
easy_install -i http://pypi.pediapress.com/simple/  mwlib mwlib.rl   pyfribidi
cd ~
git clone https://github.com/pediapress/texvc
cd texvc; make; make install

pip install ndg-httpsclient
configure services
mkdir /var/log/mediawiki/
chown www:www -R /var/log/mediawiki/
config set nserve service status enabled access localhost TCPPort 19737
config set mw-qserve service status enabled access localhost TCPPort 19252
config set nslave service status enabled access localhost

cat  >/etc/systemd/system/nslave.service <<- EOM
[Unit]
Description=Mediawiki_collection_extension nslave
After=network-online.target mw-qserve.service  nserve.service

[Service]
Type=simple
User=www
Group=www
Environment=PATH=/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin
ExecStart=/bin/bash -c 'nslave --serve-files-port=20361 --host=localhost --port=19252 --numprocs=1 --cachedir /var/log/mediawiki/ '

[Install]
WantedBy=sme-server.target

EOM

cat  >/etc/systemd/system/mw-qserve.service <<- EOM
[Unit]
Description=Mediawiki_collection_extension mw-qserve
After=network-online.target nserve.service

[Service]
Type=simple
User=www
Group=www
Environment=PATH=/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin
ExecStart=/bin/bash -c 'mw-qserve -p 19252 -i localhost '

[Install]
WantedBy=sme-server.target
EOM


cat  >/etc/systemd/system/nserve.service <<- EOM
[Unit]
Description=Mediawiki_collection_extension nserve
After=network-online.target

[Service]
Type=simple
User=www
Group=www
Environment=PATH=/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin
ExecStart=/bin/bash -c 'nserve --port=19737 --qserve=localhost:19252 '

[Install]
WantedBy=sme-server.target

EOM

/etc/e-smith/events/actions/systemd-default
/usr/bin/systemctl start nserve.service mw-qserve.service nslave.service


you will need pdftk from dag (this repo does not exist anymore so here a way to get the files) 

/sbin/e-smith/db yum_repositories set dag repository  Name 'Dag - EL6'  BaseURL 'http://www.mirrorservice.org/sites/apt.sw.be/redhat/el6/en/$basearch/dag'  EnableGroups no  GPGCheck yes  GPGKey https://www.mirrorservice.org/sites/apt.sw.be/RPM-GPG-KEY.dag.txt  Visible no  Exclude freetype,htop,iptraf,rsync,syslinux  status disabled
signal-event yum-modify
yum install pdftk --enablerepo=dag

add the repo epel

install compilation tools 

yum install gcc gcc-c++ cpp make python-devel libjpeg-devel zlib-devel freetype-devel  lcms-devel libxslt-devel libxml2-devel git ocaml

install needed dependencies 

yum install python-pip python-gevent python-py python-apipkg  ImageMagick texlive-latex texlive-texvc blahtexml --enablerepo=epel
this should install
======================================================================================================================
Installation:
 python-apipkg                           noarch              1.0-1.el6                     epel                  10 k
 python-gevent                           x86_64              0.13.8-3.el6                  epel                 189 k
 python-pip                              noarch              7.1.0-1.el6                   epel                 1.5 M
 python-py                               noarch              1.4.18-1.el6                  epel                 307 k
 texlive-latex                           x86_64              2007-60.el6_7                 base                  83 k
Dependencies:
 kpathsea                                x86_64              2007-60.el6_7                 base                 118 k
 netpbm                                  x86_64              10.47.05-11.el6               base                 802 k
 netpbm-progs                            x86_64              10.47.05-11.el6               base                 1.7 M
 openjpeg-libs                           x86_64              1.3-16.el6_8                  updates               62 k
 poppler                                 x86_64              0.12.4-12.el6_9               updates              559 k
 poppler-data                            noarch              0.4.0-1.el6                   base                 2.2 M
 psutils                                 x86_64              1.17-34.el6                   base                  46 k
 python-greenlet                         x86_64              0.4.2-1.el6                   epel                  24 k
 python-setuptools                       noarch              0.6.10-3.el6                  base                 336 k
 tex-preview                             noarch              11.85-10.el6                  base                  50 k
 texlive                                 x86_64              2007-60.el6_7                 base                 1.8 M
 texlive-dvips                           x86_64              2007-60.el6_7                 base                 193 k
 texlive-texmf                           noarch              2007-39.el6_7                 base                 2.8 M
 texlive-texmf-dvips                     noarch              2007-39.el6_7                 base                 237 k
 texlive-texmf-errata                    noarch              2007-7.1.el6                  base                 5.3 k
 texlive-texmf-errata-dvips              noarch              2007-7.1.el6                  base                 5.3 k
 texlive-texmf-errata-fonts              noarch              2007-7.1.el6                  base                 5.5 k
 texlive-texmf-errata-latex              noarch              2007-7.1.el6                  base                 5.4 k
 texlive-texmf-fonts                     noarch              2007-39.el6_7                 base                  48 M
 texlive-texmf-latex                     noarch              2007-39.el6_7                 base                 5.3 M
 texlive-utils                           x86_64              2007-60.el6_7                 base                 254 k

pip install : 

# apipkg provided in the repo is 1.4 and does not work with python 2.6
pip install  apipkg==1.2
# Pillow is required, but the version 4.0 has a syntax error when running with python 2.6
pip install -i http://pypi.pediapress.com/simple/ Pillow==3.4.2 --trusted-host pypi.pediapress.com
# last version fails with python 2.6
pip install -i http://pypi.pediapress.com/simple/ --trusted-host pypi.pediapress.com  pygments==1.6
pip install -i http://pypi.pediapress.com/simple/ mwlib mwlib.rl   pyfribidi --trusted-host pypi.pediapress.com

if the previous fails: 

easy_install -i http://pypi.pediapress.com/simple/  mwlib mwlib.rl   pyfribidi
install texvc 
cd ~ 
git clone https://github.com/pediapress/texvc
cd texvc; make; make install PREFIX=~/pp

ultimately, unless you upgrade to python 2.9 you will need a small hack on the PIL library : 

vim /usr/lib64/python2.6/site-packages/PIL/Image.py

find 

def tostring(self, *args, **kw): 
   raise Exception("tostring() has been removed. " + "Please use tobytes() instead.")

and replace it with 

def tostring(self): return self.tobytes()
startup script
mkdir /var/log/mediawiki/
chown www:www -R /var/log/mediawiki/
cat  >/etc/init.d/mw-serve <<- EOM
#!/bin/sh

PATH=/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin

case "\$1" in
        start)
                su - www -s /bin/bash -c 'nserve --port=19737 --qserve=localhost:19252  >> /var/log/mediawiki/mw-serve_log.txt 2>&1 &'
                su - www -s /bin/bash -c 'mw-qserve -p 19252 -i localhost >> /var/log/mediawiki/mw-serve_log.txt 2>&1 &'
                su - www -s /bin/bash -c 'nslave --serve-files-port=20361 --host=localhost --port=19252 --numprocs=1 --cachedir /var/log/mediawiki/ >> /var/log/mediawiki/mw-serve_log.txt 2>&1 &'
                #su - www -s /bin/bash -c 'postman --cachedir /var/log/mediawiki/ >> /var/log/mediawiki/mw-serve_log.txt 2>&1 &'
        ;;
        stop)
                #mv /var/log/mediawiki/mw-serve_log.txt /var/log/mediawiki/mw-serve_log.txt.old
                killall nserve
                killall mw-qserve
                killall nslave
                #killall postman
        ;;
        force-reload|restart)
                \$0 stop
                \$0 start
        ;;
        *)
                echo "Usage: /etc/init.d/mw-serve {start|stop}"
                exit 1
        ;;
esac

exit 0
EOM
chmod +x /etc/init.d/mw-serve
pushd /etc/rc.d/rc7.d/
ln -s  /etc/rc.d/init.d/mw-serve S80mw-serve
popd
logrotate :
cat  > /etc/logrotate.d/mwlib <<- EOM

/var/log/mediawiki/mw-serve_log.txt
{
        compress
        weekly
        rotate 4
        maxsize 5M
        dateext
        dateformat -%Y%m%d%H%M%S
        missingok
        postrotate
                /etc/init.d/mw-serve restart
        endscript



}

EOM

You might also need to do

if python complains too much of SSL and SNI 

yum install openssl-devel python-devel libffi-devel -y
pip install pyOpenSSL ndg-httpsclient pyasn1

see https://stackoverflow.com/questions/29099404/ssl-insecureplatform-error-when-using-requests-package

Sources

  1. http://mwlib.readthedocs.io/en/latest/installation.html
  2. https://stackoverflow.com/questions/36592618/reportlab-image-to-pdf-please-call-tobytes
  3. http://edutechwiki.unige.ch/en/Mediawiki_collection_extension_installation


Alternatives

use rpm for centos7 https://gitlab.com/harbottle/epypel

https://harbottle.gitlab.io/epypel/7/x86_64/

to test

mw-render --config http://en.wikipedia.org/w/ --output test.pdf --writer rl Physics

or 

mw-zip -c https://mywiki.com/ -o test.zip "bunch of pages" "here also"
mw-render -c test.zip -o test.pdf -w rl --language fr
Retrieved from "https://wiki.koozali.org/index.php?title=Mwlib&oldid=42323"