Ben Biddington

Whatever it is, it's not about "coding"

How to host a website from home

leave a comment »

My weekend project for the last couple of weeks has been to find out how to expose a website from my home computer.

And after doing so, it appears there are two requirements:

  1. Instructing firewall to forward ports
  2. Allowing requests through firewall on the target computer

I am using a SpeedTouch 585 forwarding to a laptop running Windows 7.

It is also recommended of course that you have a static ip address.

How to instruct router to forward a port

TIP: Ensure you don’t have an ip address conflict (i.e., more than one computer on the same ip address) on your network, perhaps that confuses things for forwarding. Giving whatever machine you’re forwarding to a static ip address probably makes sense — you won’t need to change anything when that machine joins network then.

I have a SpeedTouch 585, and I did it through the web interface:

  • Go to: “Home > Home Network > Devices > [device name]”
  • Select “Configure” from the upper right part of the screen (between “Overview” and “Help”)
  • From the “Connection Sharing” section, select “HTTP Server (World Wide Web)” from the list and press the “Add” button
  • You’re now forwarding all HTTP requests to your public ip address on port 80 to your selected device on port 80.

There are full also instructions for port forwarding on a SpeedTouch 585.

TIP: You can telnet straight in to router on default port (23), bypassing the web UI.

TIP: Windows 7 has telnet disabled. Search for “turn Windows features on or off” to get the applet for enabling it.

Adding new firewall rules

Firewall rules are edited and applied through the “Game & Application Sharing” section.

For example. you may wish to host a website on port 99 on your laptop:

  • Go to “Home > Toolbox > Game & Application Sharing”
  • From the “Pick a task…” section, choose “Create a new game or application”
  • From the “Clone Existing Game or Application” list, choose “HTTP Server (World Wide Web)”
  • Check “Manual Entry of Port Maps”
  • Press “Next”
  • Enter 80 and 80 as the “Port Range”
  • Enter 99 as the “Translate to”
  • That’s it

Here’s the resultant mapping:

Protocol  Port Range	Translate To ...  Trigger Protocol  Trigger Port
Any	  80 - 80	99 - 99		  -		    -

TIP: You cannot edit anything in the “Home > Toolbox > Game & Application Sharing > Game or Application Definition” list if it is assigned to something (The edit link is missing). Unassign it before you edit it.

TIP: You have to explicitly enable logging for each “Game of Application” you attach to a device, otherwise you’ll get no logging at all.

What forwarding rules really look like

In the event logs, here’s what I get when I run a test with port 80:

FIREWALL rule (1 of 1) : Protocol: TCP
Src ip: 69.163.149.200
Src port: 47603
Dst ip: 192.168.1.65
Dst port: 99
Chain: forward_host_service
Rule Id: 2
Action: accept

Even though it does display an amber light instead of green, I think this is just a forwarding record, not a warning.

That stuff about the forward_host_service chain is available via telnet. Telnet in to your router and open the chains list.

Here are the chains I have:

Name                                             Description
-----------------------------------------------------------------
sink                                             system
forward                                          system
source                                           system
sink_fire                                        system
forward_fire                                     system
source_fire                                      system
forward_host_service                             system
forward_level                                    system
sink_system_service                              system
forward_multicast                                system
forward_level_BlockAll                           system
forward_level_Standard                           system
forward_level_Disabled                           system

Which includes forward_host_service, which contains these rules (including number 3):

:firewall rule list chain=forward_host_service format=cli

:firewall rule add chain=forward_host_service index=1
    name=_f_u_192.168.1.65:55768_55768
    dstip=_u_192.168.1.65
    serv=_u_17:55768_55768
    log=disabled
    state=enabled
    action=accept

:firewall rule add chain=forward_host_service index=2
    name=_f_sv_192.168.1.65:80_80
    dstip=_sv_192.168.1.65
    serv=_sv_6:99_99
    log=disabled
    state=enabled
    action=accept

:firewall rule add chain=forward_host_service index=3
    name=_f_sv_192.168.1.65:3128_3128
    dstip=_sv_192.168.1.65
    serv=_sv_6:3128_3128
    log=enabled
    state=enabled
    action=accept

How to allow connections through your firewall

Depends on your vendor obviously, but I am running ESET and I did this:

  1. Setup > Personal firewall > Configure rules and zones…
  2. Add new rule: TCP & UDP, port 80, All remote ports, All addresses

Coping with rejection

If you’ve gotten this far, then your router is forwarding as expected, but the test may still be failing. The next place to look then is on the machine being forwarded to.

Inspect the firewall logs on the target machine and you should have something like:

Packet blocked by active defense (IDS)
192.168.1.65:99
69.163.149.200:41791
TCP

So I had to add a new firewall rule.

If you have problems with your rules, put the firewall into interactive mode (this will prompt you to allow or deny all incoming requests), run the test, accept the connection and inspect the rule it generates.

TIP: Take care with the executable, try the rule with and without to see if it makes a difference.

How to test your router is open on a particular port

Try this utility.

Troubleshooting

Where are the SpeedTouch router log files?

Home > SpeedTouch > Event Logs

References

Written by benbiddington

4 May, 2010 at 13:37

Whose leg do I have to hump to get a Facebook developers forum account?

with 4 comments

How does one go about registering for the Facebook developer forum?

Filling out the form

Produces “Could not open socket” error in Chrome and Firefox.

Facebook Connect

On the register screen there is a Facebook Connect option, but this does nothing in Chrome, and does not appear at all in Internet Explorer.

Firefox does produce the window, so I select yes or whatever but then I can still not sign in.

Then I ask for my password to be sent, only to be told there is no account for my address.

Okay, then I think perhaps I don’t need to sign in to access the forum. I sign in to facebook and revisit: still can’t do anything but read posts.

In other news

Interestingly I do have punBB listed on my Application settings screen.


Written by benbiddington

29 April, 2010 at 13:37

Posted in development

Tagged with , , , ,

2010-04-04 — new signature

leave a comment »

My customary signature has officially changed from:

<bb />

to:

<bb>
    <mb />
</bb>

Many thanks and kind regards.

Written by benbiddington

28 April, 2010 at 13:37

Posted in development, diary

Tagged with , , ,

Adobe Content Server — packaging large files is painful

with 5 comments

Ordinarily, working with Adobe Content Server (ACS) is more or less tolerable, but recently we have encountered what may be another indication of the quality of this product.

Packaging

Packaging an ebook amounts to posting a signed request to the server, describing the file you’d like to ingest.

You can use UploadTestJar (which may have its own issues), or (if you’re lucky) you can rewrite some sample codes in your chosen language and use that.

You’d think then, that once you’ve got it working you can summarily be on your way, forget about ACS and finish your application.

And you can.

Until the OutOfMemoryExceptions start

While test-driving our application, we naturally wanted to describe what happens with different sizes of files, so we tried some large ones. These would fail with errors about being out of heap space, errors like:

21-Apr-2010 13:37:26 org.apache.catalina.core.StandardWrapperValve invoke
at com.adobe.adept.servlet.AdeptServlet.doPost(AdeptServlet.java:180)
SEVERE: Servlet.service() for servlet Package threw exception
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Unknown Source)
at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
at java.lang.AbstractStringBuilder.append(Unknown Source)
at java.lang.StringBuffer.append(Unknown Source)
at com.adobe.adept.xml.XMLAbstractDigestSink.characters(XMLAbstractDigestSink.java:133)
at com.adobe.adept.xml.XMLSink.characters(XMLSink.java:261)
at com.adobe.adept.xml.XMLFieldReader.characters(XMLFieldReader.java:447)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.characters(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
at com.adobe.adept.servlet.AdeptServlet.doPost(AdeptServlet.java:180)

We tried adjusting the available memory for “sags” (as Dan Rough would put it), and this did work to a certain degree, but is not a satisfactory solution.

To me it looks a bit like an attempt to load entire file into memory at once. Surely this can’t be right, can it?

Examining AdeptServlet

In an effort to understand the nature of the problem before solving it, we decided to have a look at that servlet. We decompiled it and set about finding that class, and it is:

com.adobe.adept.packaging.servlet.Package

In the doPost method, there are these lines:

if (paramParsedRequest.data != null) {
    localObject1 = new PDFPackager(paramParsedRequest.data);
} else {
    localObject1 = new PDFPackager(new File(paramParsedRequest.dataPath));
}

This shows the two methods of loading a PDFPackager.

Examining PDfPackager, we can see the ctor has four overloads including these two:

public PDFPackager(byte[] paramArrayOfByte) throws Exception {
    this(new ByteBufferByteReader(paramArrayOfByte));
}

public PDFPackager(File paramFile) throws Exception {
    this(new FileInputStream(paramFile));
}

So, it appears the problem may result from usage of the first version.

That  paramParsedRequest argument to doPost is of type ParsedRequest, and its data property is a Byte array.

This could be a problem: when submitting a package request with a data node instead of a dataPath node, we’re using the byte array overload.

Where is the error actually coming from?

From the stacktrace it looks as though it is coming from whatever is creating the arguments to supply to Package.doPost.

This is the responsibility of Package‘s supertype: AdeptServlet<RequestParser>. It is this class that is responsible for parsing the http request into one of those ParsedRequest objects, and then supplying that to Package.doPost.

The problems starts here at the top level request handling:

// AdeptServlet<RequestParser>
doPost(HttpServletRequest paramHttpServletRequest, HttpServletResponse paramHttpServletResponse)

This is where the request parsing happens, and then — as the stack trace shows — an error ends up resulting from XMLAbstractDigestSink.characters.

XMLAbstractDigestSink.characters attempts to append data to an internal StringBuffer.

Summary

This mechanism has not been designed in any kind of scalable manner — buffering files in memory is utterly nuts.

Why not just write the posted data to a temp file and use the other PDFPackager ctor?

The solution

Well, one suggestion is to not post the files at all, but make a slightly different packaging request that supplies a path to a file on disk rather than the file itself.

To do so requires — as described in ContentServer_Technical_Reference.pdf — supplying a dataPath node in your request instead of a data node.

The downside for us is that now we need to manage this shared file location — a non-trivial task when working with Windows services.

Another (unlikely) solution

Modify the application, i.e., AdeptServlet<RequestParser> so it first copies the posted file to disk, and then proceeds as though it received a dataPath request.

Pretty hard without the source — it’s probably actually against the law, is it?

References

Written by benbiddington

27 April, 2010 at 14:00

Windows services and net use

leave a comment »

We have some Windows services that need to access network shares, and even though we have net used, those resources are still unavailable. It appears this is because our services are running as LocalSystem.

How to check the connections available to LocalSystem

1. Open command prompt as LocalSystem

Follow these instructions to get a LocalSystem cmd prompt using at.exe.

Note: You can use at.exe only when the Schedule service is running, to find out:

sc query schedule

2. List connections

net use

You will see the set of connections available.

Note this set is different to the list generated by ordinary command prompt (your account).

How to add connection for LocalSystem

Don’t know, that method is not very automatable.

References

Written by benbiddington

27 April, 2010 at 13:37

Posted in development

Tagged with , , , , ,

Async operations and exceptions

leave a comment »

We have had the case where we’re creating a class that allows clients to block while internally it reads an entire stream asynchronously. This class encapsulates the state required to perform such a task.

While attempting to write unit tests for exceptions, we found that an exception thrown during the asynchronous operation would not be thrown to client. Debugging showed that the exception was being thrown, but no notification was being sent to the parent thread.

No such thing as unhandled exceptions on managed threads

[MSDN] [since .NET Framework v2.0] There is no such thing as an unhandled exception on a thread pool [or finalizer] thread. When a task throws an exception that it does not handle, the runtime prints the exception stack trace to the console and then returns the thread to the thread pool.

Errors raised on a child thread are essentially lost when the thread exits. This means there is some work required to propagate these exceptions.

This requires a blocking wait on the part of the client, and a mechanism for storing the exception so the parent thread can read it.

As an example, we have implemented an AsyncStreamReader which contains a blocking ReadAll method. If an asynchronous read fails with an exception, that exception is exposed internally as a field, and the waiting thread is then signalled. Once the waiting thread wakes up it checks the exception field and throws it if required.

We have blocking Read operation that waits for an async read to complete. The notification mechanism is a ManualResetEvent (WaitHandle).

  1. T1: Invoke ReadAll.
  2. T1: Start async operation (spawns T2).
  3. T1: Wait.
    1. T2: Async operation encounters exception.
    2. T2: Store exception in _error field.
    3. T2: Signals T1.
    4. T2: Returns without triggering any subsequent reads.
    5. T2: Thread exits
  4. T1: Parent thread resumes (still inside ReadAll).
  5. T1: Checks _error field. If it is not null, throw it, otherwise return.
  6. T1: Exception is now propagated

References

Written by benbiddington

27 April, 2010 at 13:37

Scala — Futures

leave a comment »

A future is a placeholder for the return value of an asynchronous operation, it’s left to clients to decide when to block and wait for reply value.

It is an alternative to blocking on receive.

For example, the double-bang on Actor causes operation to return a future:

   /**
   * Sends msg to this actor and immediately
   * returns a future representing the reply value.
   */
  def !!(msg: Any): Future[Any] = {
    val ftch = new Channel[Any](Actor.self)
    send(msg, ftch)
    new Future[Any](ftch) {
      def apply() =
        if (isSet) value.get
        else ch.receive {
          case any => value = Some(any); any
        }
      def isSet = value match {
        case None => ch.receiveWithin(0) {
          case TIMEOUT => false
          case any => value = Some(any); true
        }
        case Some(_) => true
      }
    }
  }

Which can then be used to obtain the reply.

The Future class takes an InputChannel as its ctor argument. This channel is monitored to determine the future’s completion status.

In this instance, the future by !! is configured with the reply channel as supplied to send. In short, the actor has sent itself a message and specified that the future’s channel should be notified when complete. The future then just monitors that channel for the reply.

Note: Actor.send invokes the act method using a Reaction, which spawns threads and runs actors.

Only the actor creating an instance of a Channel may receive from it. This means that the future here must be running on the same thread as the actor that created it.

The send call is instructing the reply to be returned to the channel being monitored by the future.

Why does future block until actor returns value?

This is because it blocks on the channel, waiting for reply:

...
def apply() =
    if (isSet) value.get
    else ch.receive {
        case any => value = Some(any); any
    }
...

and Channel.receive is a ultimately a blocking operation, since it invokes receive on the actor it belongs to:

...
def receive[R](f: PartialFunction[Msg, R]): R = {
    val C = this.asInstanceOf[Channel[Any]]
    recv.receive {
        case C ! msg if (f.isDefinedAt(msg.asInstanceOf[Msg])) => f(msg.asInstanceOf[Msg])
    }
}
...

Note that recv here is the Actor supplied in Channel ctor.

Consider this example:

val aFuture = future[String] {
    currentThreadId
};

Internally, a new actor is created and has its double bang invoked:

def future[T](body: => T): Future[T] = {
    case object Eval
    val a = Actor.actor {
        Actor.react {
            case Eval => Actor.reply(body)
        }
    }
    a !! (Eval, { case any => any.asInstanceOf[T] })
}

And by examining the apply method above, we know this blocks until a message is received from channel.

Written by benbiddington

24 April, 2010 at 13:37

Posted in development

Tagged with , , ,