Announcement

**jmj09** · January 12, 2016, 02:14 PM

Shifting to Nginx /2

I shall try to explain there the general scheme

Code:

First came the Wild WAN
||||||||||||||||||||||||
First funnel is the router which allows only ports 80 and 443
     |         |   only 2 ports remain open to my fixed ip
    80       443   ports are transfered to nginx server by port management in the router.
Second funnel is in first block of code from nginx .conf which rewrites http to https
                |
              443 only remains.
Then the query from internet is parsed to see if it is static stuff
Then if it is HS3 page
Then json api calls
Then rss calls (my own api)

Static files are served straight away 
aiming at port 443 (ssl) with max cache

Some variable files (eg. jpeg images from my ipcam)
aiming at port 443 with no cache

HS3 pages are served through a reverse proxy (internal to nginx) 
aiming at HS3 server on port 81 (not accessible from outside)

json calls are served straight
aiming at port 81 (the one from HS3 server)

rss calls are served through a reverse proxy 
aiming at my own nodejs server on port 1000 with no cache

No pasword for local port is on in HS3 setup.

Please see config here after

**jmj09** · January 12, 2016, 02:15 PM

Shifting to Nginx /3

nginx.conf file
This is the file where you handle the behaviour of your server.
I shall add more comments if there is questions about pieces of code.

Code:

worker_processes  1;
error_log  'C:\\Program Files (x86)\\nginx\\logs\\error.log';
pid        'C:\\Program Files (x86)\\nginx\\logs\\nginx.pid';
events {
    worker_connections  1024;
}

http {

  include       mime.types;
  default_type  application/octet-stream;
  sendfile      on;
  keepalive_timeout  65;

	# enable gzip compression
    gzip  on;
    gzip_static on;
    gzip_comp_level 6;
    gzip_min_length 1400;
    gzip_types mime.types;
    gzip_buffers  4 32k;
    gzip_vary  on;
    gzip_http_version 1.1;
    gzip_disable "MSIE [1-6]\.(?!.*SV1)";
	# end gzip configuration
  
  #rewrites http to https
  server {
    listen      80;
    server_name your.server.com;
    rewrite     ^   https://$server_name$request_uri? permanent;
  }
  
  #Main https server job default port 443
  server {  
    listen  443 ssl;
    server_name  your.server.com;
    ssl_certificate 'C:\\Program Files (x86)\\nginx\\ssl\\your.server.com.crt';
    ssl_certificate_key 'C:\\Program Files (x86)\\nginx\\ssl\\your.server.com.deprotected.key';
    root 'C:\Program Files (x86)\HomeSeer HS3\html';
    add_header X-Whom direct;
    auth_basic  "Acces reserve";
    auth_basic_user_file  'C:\\Program Files (x86)\\nginx\\conf\\.htpasswd';  
    satisfy any;
    allow 127.0.0.1;
    #allow 192.168.0.10;
    deny  all;
    
    #Errors -> Locations fancy own error page
    error_page 404 /error_404.html;
      
 
	  #Prevents hidden files (beginning with a period) from being served
	  location ~ \/\. { access_log off; log_not_found off; deny all; }

    #serve homeseer files the ones without dot in the path
    location ~* (?<![\.\/]...)$ {
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;      
      proxy_pass http://127.0.0.1:81 ;
      add_header X-Whom HS3;
      expires -1;
    }

    #serve static files 
    location ~* ^.+\.(?:css|cur|js|png|gif|htc|ico|txt|xml|otf|ttf|eot|woff|svg)$ {
      try_files $uri $uri/ /error_404.html;
      add_header X-Whom all;
      add_header Cache-Control public;
      expires max;
    }

    #serve jpeg, csv, html variables files including jpeg from my IPcam
    location ~* ^.+\.(?:jpe?g|csv|html)$ {
      try_files $uri $uri/ /error_404.html;
      add_header X-Whom vary;
      expires -1;
    }

    # serve HS3 json api via proxy
    location ~* \/(JSON|json) {
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_pass http://127.0.0.1:81 ;
      add_header X-Whom json;
      expires -1;
    }

    # serve rss via proxy, this is my own api (express node js) to deal with csv files
    location ~* \/rss {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_pass http://127.0.0.1:1000 ;
      add_header X-Whom rss;
      expires -1;			
    }

  }

}

**ZoRaC** · February 22, 2017, 02:04 PM

I just implemented this myself - thanks, it works great!

I also extended it to give med an encrypted connection for Geofence with the PHLocation-plugin.

**Moskus** · February 28, 2017, 05:33 AM

This works!

But I have problems serving asp pages when the asp page is specified. I use Jon00s Quick page builder (with the ASP option as it uses much less RAM). If I set it up as as the loading page, then it is displayed without problems.

If I specify the filename, it doesn't work.

So this url:
https://myserver/ <- works, and linking to a asp page
https://myserver/buildpage.asp <- doesn't work!
https://myserver/buildpage.aspx <- works!

What am I not getting?

**jmj09** · May 23, 2017, 03:46 AM

Since the begining of my post, things evolved...
Let's encrypt came, my skill enhanced, though I still don't wanna cry...

Thus I am now in my own domain name, with better nginx conf file I hope.

Let,s encrypt certificates are a pain to manage (for me) on the HS w10 server, I did not succeed in automating the process.

Regular expression are at least as efficient as sudoku to protect your brain from aging...

I do not fully understand the deny/allow priorities...

You can get ratings about ssl security here https://www.ssllabs.com/ssltest/analyze.html.
With the following conf file I get a "A" which is not optimum, I think you can reach "A+" in this rating

Comments Welcome as usual, specially security issues that could be detected in my conf file.

Jean

Code:

worker_processes  1;

error_log   'C:\\nginx\\logs\\error.log debug';
pid         'C:\\nginx\\logs\\nginx.pid';

events {
  worker_connections  1024;
}

http {
  #csp
    server_tokens off;
    add_header X-Frame-Options SAMEORIGIN;
    add_header X-Content-Type-Options nosniff;
    add_header X-XSS-Protection "1; mode=block";
    add_header Content-Security-Policy "default-src 'self'; script-src 'self'; img-src 'self'; style-src 'self' ; font-src 'self'; child-src 'none'; object-src 'none'";
    ssl_session_cache shared:SSL:50m;
    ssl_session_timeout 5m;
    # enables server-side protection from BEAST attacks
    ssl_prefer_server_ciphers on;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    # ciphers chosen for forward secrecy and compatibility
    ssl_ciphers "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4";
    # enable ocsp stapling (mechanism by which a site can convey certificate revocation information to visitors in a privacy-preserving, scalable manner)
    resolver 8.8.8.8;
    add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";
  #end CSP

  include       mime.types;
  default_type  application/octet-stream;
  sendfile      on;
  keepalive_timeout  65;

	# enable gzip compression
    gzip  on;
    gzip_static on;
    gzip_comp_level 6;
    gzip_min_length 1;
    gzip_types "*";
    gzip_buffers  4 32k;
    gzip_vary  on;
    gzip_http_version 1.1;
    gzip_proxied any;
	# end gzip configuration

  server {
    listen 80;
   #redirect all to https
    rewrite_log on;
    server_name myown.domain.com;
    return 301 https://myown.domain.com$request_uri;
  }

  #Main https server job default port 443
  server {
    listen  443 ssl;
    ssl on;
    rewrite_log on;
    server_name myown.domain.com;
    ssl_certificate 'C:\\pathToNginx\\ssl\\fullChain.pem';
    ssl_certificate_key 'C:\\pathToNginx\\ssl\\privateKey.pem';
    root 'C:\Program Files (x86)\HomeSeer HS3\html';
    add_header X-Whom direct;
    auth_basic  "closed website";
    auth_basic_user_file  'C:\\pathToNginx\\conf\\.htpasswd';  
    satisfy any;
    allow 127.0.0.1;
    allow 192.168.0.1;
    allow 192.168.0.4;
#single and subnet deny ips selected from frequent raiders (out of logs)
    deny 198.0.236.0/24;
    deny 194.28.115.231;
    deny 185.159.36.0/24;
    deny 185.159.37.0/24;
    deny 176.157.128.233;
    deny 40.77.167.0/24;
    deny 89.163.242.226;
    deny 188.0.236.176;
    deny  all;

    location @404 {
      return 302 '\\error_404.html';
    }
    error_page 404 = @404;

	  #Prevents hidden files (beginning with a period) from being served
	  location ~ \/\. { access_log off; log_not_found off; deny all; }

    #serve homeseer reqs
    location ~* \/JSON\? {
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 
      proxy_intercept_errors on;      
      proxy_pass http://127.0.0.1:81 ;
      add_header X-Whom JSON;
      expires -1;
    }
    
    #serve homeseer files the ones without dot, = and ? in the path
    location ~* ^[^=?.]*$ {
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 
      proxy_intercept_errors on;      
      proxy_pass http://127.0.0.1:81 ;
      add_header X-Whom HS3;
      expires -1;
    }
    
    #serve static files 
    location ~* ^.+\.(?:css|cur|js|png|gif|htc|ico|txt|xml|otf|ttf|eot|woff|svg)$ {
      try_files $uri /error_404.html;
      add_header X-Whom Static;
      add_header Cache-Control public;
      expires max;
    }

    #serve jpeg, csv variables files including jpeg from my IPcam
    location ~* ^.+\.(?:jpe?g|csv|html)$ {
      try_files $uri /error_404.html;
      add_header X-Whom vary;
      expires -1;
    }

    # serve rss via proxy, this is my own api (express node js) to deal with own json files
    location ~ \/=(rss|ver|mem|cpu|osc|pas)$ {
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_intercept_errors on;
      proxy_pass http://127.0.0.1:1000 ;
      add_header X-Whom api1000;
      expires max;			
    }    
  }
}

**Kerat** · May 23, 2017, 08:52 PM

Shifting to Nginx /1

Very well written post. I locked down and centralized all the internal resources I wanted to make available on the pubic Internet using squid's reverse proxy and a let's encrypt ACME client. I opted to host my solution on my PFsense firewall using readily available packages that run in the PFSense system (squid reverse proxy and ACME).

I never made the connection that you could run scripts this way. Would you confirm that you are running scripts within node.js outside of NGINX?

Sent from my iPhone using Tapatalk

**jmj09** · May 26, 2017, 10:18 AM

Hi Kerat
As soon as you can build a running server you can proxy it with nginx. Could be Python, Ruby on rails and nowadays of course node js.

Here after, code for a node js server on port 1000, which answers for several metrics in parallel with HS3. My lib for file and error management is lacking, this is to give a glance of easy scripting

Of course you can embed HS3 and nodejs through Json hs3 api calls.
At this time all my csv files are built with nodejs (json calling HS3) and no longer HS3 .net, no more memory trouble and system is stable. Node js is easy to debug. I am using pm2 for app management.

Jean

Code:

const five = require('take-five');
const os = require('os');
const exec1 = require('child_process').exec;
const exec2 = require('child_process').exec;
const util = require('util');
const myF = require('./noisetfunc.js');
const server = five();

//tested in test.js
// different metrics on host cpus see node docs
server.get('/=osc', function(req, res) {
  try {
    const osCpus = 	os.cpus();
    res.send(osCpus);
  } catch(err) {
    myF.processError('api1000-server.get-osc ', err, 55);
    res.send([{"error":"api1000-server.get-osc"}]);
  }
});

//tested in test.js
server.get('/=ver', function(req, res) {
  try {
    const verRes = process.versions;
    res.send(verRes);
  } catch(err) {
    myF.processError('api1000-server.get-ver ', err, 55);
    res.send([{"error":"api1000-server.get-ver"}]);
  }
});

//get atmospheric pressure from a fine yocto sensor
server.get('/=pas', function(req, res) {
  const options = {
    timeout: 5000,
    killSignal: 'SIGKILL'
  };
  try {
    exec2('YPressure METEO get_currentValue', options, function (err, stdout) {
      const line = stdout.toString();
      let results = line.slice(-11,-3);
      isNaN(results) && (results = 0);
      res.send(results);
    });
  } catch(err) {
    myF.processError('api1000-server.get-pas ', err, 45);
    res.send(0);
  }
});

//
server.get('/=rss', function (req, res) {
  const options = {
    timeout: 5000,
    killSignal: 'SIGKILL'
  };
  exec1('tasklist', options, function (err, stdout) {
    const lines = stdout.toString().split('\n');
    const results = [];
    try {
      lines.forEach(function (line) {
        const proc = line.substring(0, 26),
          pid = line.substring(26, 35),
          util1 = line.substring(66, 69),
          util2 = line.substring(70, 73);
        results.push([proc, pid, util1 + util2]);
      });
      res.send(results);
    } catch(err) {
      myF.processError('api1000-server.get-rss ', err, 55);
      res.send([{"error":"api1000-server.get-rss"}]);
    }
  });
});

//tested in test.js
server.get('/=cpu', function(req, res) {
  try {
    const CPUout = process.cpuUsage();
    const CPUUSER = + (parseFloat(CPUout.user) / 1000000).toFixed(2);
    const CPUSYST = + (parseFloat(CPUout.system) / 1000000).toFixed(2);
    const CPUFormat = {
      user: CPUUSER ,
      system: CPUSYST
    };
    res.send(CPUFormat);
  } catch(err) {
    myF.processError('api1000-server.get-cpu ', err, 55);
    res.send([{"error":"api1000-server.get-cpu"}]);
  }
});

//tested in test.js
server.get('/=mem', function(req, res) {
  try {
    const memRes1 = util.inspect((process.memoryUsage().rss / 1048576));
    const memRes2 = util.inspect((process.memoryUsage().heapTotal / 1048576));
    const memRes3 = util.inspect((process.memoryUsage().heapUsed / 1048576));
    const memRes = {};
    memRes.rss = + parseFloat(memRes1).toFixed(2);
    memRes.heapTotal = + parseFloat(memRes2).toFixed(2);
    memRes.heapUsed = + parseFloat(memRes3).toFixed(2);
    res.send(memRes);
  } catch(err) {
    myF.processError('api1000-server.get-mem ', err, 55);
    res.send([{"error":"api1000-server.get-mem"}]);
  }
});

server.on('uncaughtException', function(req, res, route, err) {
  console.log('erreur api1000 ligne 90 : ' + err);
  res.send(500, '0');
});

server.listen(1000);

**beerygaz** · June 12, 2017, 07:40 AM

Hi folks, I stumbled on this thread with a lot of interest. 2 questions for those more knowledgeable on Nginx config than I am:

1. Are you able to pass the original source IP to HS3 so that it knows whether to authenticate or not? All my efforts seem to pass the proxy address to HS3 so it thinks all request come from the local network and does not try to authenticate.

2. Could this be run on on a different server? I see the "root C:\..." setting, which obviously would need to be different if nginx was running elsewhere?

**Kerat** · June 12, 2017, 09:13 AM

Shifting to Nginx /1

I know that in HAproxy for PFsense the options exist for IP pass through, but I still haven't worked it out yet either.

Edit: in my settings there is an option for "forward for" which appears to indicate that it adjusts the header to load the endpoint's IP address. This didn't do the trick for me. In the backend for my HS3 instance there is a setting for transparent clientIP that did the trick. Now when I login from work I am seeing my work's public IP address.

Sent from my iPhone using Tapatalk