Thursday, November 1, 2012

Python Ternary Operator

I was checking webmasters and noticed a search for python lambda if else that lead to here. I figured if Google was already pointing in this direction, I may as well make a post since I already know how to use an if/else in Python.

The first thing to note is that you use a ternary operator. Ternary operators are relatively recent to Python and most people I know don't seem to like ternary operators. The reason we need to use a ternary operator is because a lambda function must return a value and ternary operators always return a value. Their very purpose is to return a value based on a condition for things like variable assignments or returns on a function. The syntax for a ternary operation in Python is as follows (with variable assignment).

x = valueIfTrue if someCondition else valueIfFalse

So you can have something along the lines of this.

x = 1
print "It is one" if x == 1 else "It is not one"
# It is one

It can also be used as a single argument for a function. Due to this being a one-line statement that always returns a value, it can be used to preform a simple if/else within a lambda function.

even = lambda x: False if x % 2 else True

Now like magic, we created a lambda function for even numbers. This can be used for filtering or mapping a list, however using list comprehension is faster and does not require creating a lambda function, not to mention with a little alteration you can have a generator expression.

So there's a quick and dirty introduction to ternary operators in Python and how to evaluate an if/else in a lambda function.

Wednesday, October 17, 2012

Microdata API in XML (XHTML)

After running into an issue with the Microdata API conflicting with with a script on a site, I have looked into it recently. While I do not fully understand what the usage of it is, I decided to at the very least try to learn the syntax so that I can get past that for when it becomes something relevant and in heavy usage.

So to do so, I am looking at implementing some Microdata on my website. Now the first problem I ran into was the description of the syntax conflicts with proper XML. The conflict is an attribute without a value, itemscope. So any attempt to use this results in an XML parser error (because I server the site as an xml file, not an html file). So after a quick Google search, a solution was found in the form of a solution for another problem someone was having. So to set an itemscope in xhtml, you do this.

<div itemscope="itemscope">

Like magic, the browser knows what to do. I did not realize that I have actually done with before when using selected="selected" on a dropdown before.

The idea behind this whole Microdata API seems like a nice way to organize and access data using the DOM a lot quicker without the need for items to be sibling, parent or child nodes and offering a quick named reference without a million different ID's. However due to it being new, if used a lot, there are probably going to be a lot of hack jobs to emulate the thing. All well, hope this helps anyone looking into it but don't have the desire to use regular old HTML.

Tuesday, October 16, 2012

MyBB Post BBCode Button Problem

I wanted to post this a couple days ago when the problem was brought to my attention at a forum I code for, but was too busy to get it up until now. The problem has a fix for it, here. I just want to go over quickly what is the cause of the problem and let people know it may effect Chrome and Opera. The fix was simply changing some variable names, so it wasn't anything too complicated.

So Firefox implemented something called Microdata API. This is a new thing for HTML5 that adds more information to your markup. The catch is, whether or not you are using HTML5, it still effects your site. It adds new DOM attributes that MyBB 1.6.8 is using. This particular one is itemData. That is now an object and due to the way the script was made for the post editor, it just screws it up. From what I've read, Chrome and Opera will be implementing them soon. I hope eventually it will be made to only effect HTML5 pages.

I have yet to read up how to use the Microdata API, but if you are interested, you can find notes on it here.

Monday, October 8, 2012

Legacy Support Rant

Sometimes legacy support is good. It can lead to stable ways to develop things, easier transitioning and all around make life easier. One thing I have been noticing is a very strong concern for websites to have legacy support. The problem with this legacy support is people seem to be holding off on all the cool technologies that are being made for newer web design that can make things safer, quicker and easier. While I understand it's not always best to rely on these new technologies for safety, I use them. I do not use them for the safety, but the enforced strictness that can lead to a proper and less erratic design.

When I started my adventure into learning web design, it was 2010. I was learning XHTML 1.0. Guess what? It was new in 2001, and now it was just used as common place despite development of HTML5. This really isn't a problem, but I still saw some sites using HTML 4.01. This is even older, from 1999. This may not seem like a lot of time, but in Internet time that is rather old. Other legacy support includes a fix for png transparencies because old versions of IE don't support that, checking for addEventListener in javascript because old IE used attachEvent and many other javascript, as well as css, hacks to get around issues on older browsers.

This is all fine and dandy, however I feel people really need to move on with this stuff. I should not need to worry about support for IE5 when IE9 is already out and available for free to anyone who already has Windows. Also, with Firefox, Chrome and Opera all being free and well up-to-date with all the latest web technologies, there should not be a problem. The newer browsers are becoming more consistent (although some do things I don't like, mainly webkit) and are supporting newer and better technologies that are removing the need for flash and Java applets, which is a good thing. Why is this good? Because they can be even more of a mess to work with and they crash or lag my browsers so often that it just irritates me to no end. Some flash and java apps are cool, like flash games, some programs I used in school on blackboard and I've even seen an online photo editing program that was way better than MS Paint.

My main problem is there is a time and a place when to use each technology. When done correctly, you get a very nice website, good experience and overall ease of use with everything. When they've gone wrong is when I go to a site with about 50 flash apps all overlapping and clipping each other, lag so bad everything on the site stutters when I try to scroll and my bandwidth cluttered by pages of CSS hacks just so they could kind of get the site to look normal despite the flash apps breaking it because you can't style them with CSS and have to rely on the design of flash to work it all out.

Web design needs to start letting go of legacy support. I made my website in XHTML5 served as XML, and not all browsers can open an XML file. Also, I'm using CSS3 and "use strict" in my javascript.

Outside of web design, I see people use old code in their programs. Things that have better ways of doing them that were made. Just looking at code in a broad spectrum, even today, most business programs are still COBOL programs. I tried learning COBOL. The need to write things out in caps, count spaces so code can run correctly and in general lack simple control structures and functions that I use in other programming was a nightmare. However, in contrast, old languages like Lisp still have their place. There are some old languages out there that when I see, I have a fascination with them because of how old they are and some have things you just can't do with really anything else. You can emulate it, but its true nature only resides in itself (mainly looking at lisp and their macros).

Legacy can be fun, interesting and great for learning. However, looking at things practically, we keep things alive for far to long. There can even be an argument made about IPv4.

I suppose what I am getting at is legacy, while it is cool and useful, should not be supported so heavily in the mainstream to this point of reliance. Push for some progress, it's not abandoning the old ideas, it is simply making them better. So update once in a while.

Saturday, October 6, 2012

Strange stuff in Python (mutable objects)

I was fiddling around with Python trying to finish up a project when I noticed something strange. Now if you did not know, any mutable object you pass in Python passes a pointer to the object rather than a copy. While this does not seem all too interesting, this leads to the ability to pass something like a dictionary to a function and that function can then alter the dictionary. Thus anything after that that accesses it will see the changes, value returned or not.

Another thing about Python is global variables are stored as a dictionary, which can be obtained using the globals() function, keep in mind it returns a dictionary which means it points to the actual object, not just cloned data. Okay, so now what can we do with this? Well the globals updates whenever a global object is created, including variables. So what happens if we assign a variable to point to the globals? We get something like this (on idle).

>>> x = globals()
>>> x
{'__builtins__': <module '__builtin__' (built-in)>, '__file__': '/usr/bin/idle-python2.7', '__package__': None, 'x': {...}, '__name__': '__main__', 'main': &tl;function main at 0x1b6a410>, '__doc__': None}
>>> x["x"]
{'__builtins__': <module '__builtin__' (built-in)>, '__file__': '/usr/bin/idle-python2.7', '__package__': None, 'x': {...}, '__name__': '__main__', 'main': <function main at 0x1b6a410>, '__doc__': None}
>>> 

So what you should notice is that x and x["x"] appear the exact same because they are the exact same thing. We can dive forever into this and we will always get the same result. So what would happen if we did something that while during execution altered this? If you did not know, creating a variable in a loop in Python does not make the variable local to the loop like it would in C, it creates a global (something I hate but have learned to live with by just reusing a garbage variable that I won't use anywhere else). So the end result gives us this.

>>> for a in x: print a

__builtins__

Traceback (most recent call last):
  File "<pyshell#4>", line 1, in <module>
    for a in x: print a
RuntimeError: dictionary changed size during iteration
>>> 

Okay, so now things get interesting because we can break stuff. So let's see that again.
>>> for a in x: print a

a
__builtins__
__file__
__package__
x
__name__
main
__doc__
>>> 

Okay, so what just happened? Well, since the first call to a created the global a, the size changed. But now when we run it again, a already exists so the size does not change. Despite the values changing, it does not care as long as the length of the item is not modified. This means even in this case, we can still alter the dictionary mid-loop so long as we do not change the length. Try this out yourself then view x. You should find that a is now the last item from the loop. In fact, prior to the RuntimeError, a was still assigned the variable, meaning we can also track the value when the loop terminated. Now let's delete a from there and get a little craftier to see what happens.
>>> del a
>>> while 1:
 try:
  for a in x: print a
 except RuntimeError:
  print "Failed at " + a
 else:
  break

 
__builtins__
Failed at __builtins__
a
__builtins__
__file__
__package__
x
__name__
main
__doc__
>>> 

Now like magic, you see it failed and tried again. Now let's try one more thing, promise it's the last thing to try for now, and this one is weird. I also have no idea what happens so I ran it twice. Check it out.
>>> del a
>>> while 1:
 try:
  for a,b in x.items(): print a,b
 except RuntimeError:
  print "Failed at " + a + "=" + b
 else:
  break

 
__builtins__ <module '__builtin__' (built-in)>
__file__ /usr/bin/idle-python2.7
__package__ None
x {'a': 'x', 'b': {...}, '__builtins__': <module '__builtin__' (built-in)>, '__file__': '/usr/bin/idle-python2.7', '__package__': None, 'x': {...}, '__name__': '__main__', 'main': <function main at 0x1b6a410>, '__doc__': None}
__name__ __main__
main <function main at 0x1b6a410>
__doc__ None
>>> 

So what happened? To be perfectly honest, I have no idea what is going on, but it seems either the unpacking or the function call somehow changes the flow enough that no failure occurs. I do not know that much about the guts of Python and maybe this information is completely useless and well know. Either way, I found it interesting and if you have any cool ideas of what can be done with this or maybe know a bit more about how it works than I do, let me know.

Friday, September 28, 2012

Haskell Network not found (Ubuntu)

So for a bit recently, I've been looking into Haskell. I really want to get into some functional programming. On thing I like doing is network programming because it actually gives me something extra to interact with than my own I/O or bugging a friend to help out or check something out. So I gave a quick glance at sockets and wanted to just see what the types look like on ghci, and of course it cannot find anything related to the network package.

> import Network.Socket

:
    Could not find module `Network.Socket':
      it is not a module in the current program, or in any known package.
Leaving GHCi.
$ ghc-pkg field network exposed-modules
ghc-pkg: cannot find package network

After a few minutes of banging my head against my desk, I say screw the gui package manager and just poke around with apt-cache search. Manage to find the library and for some reason, it was not installed by default. So here's what lead me to the package and the quick and easy fix for it.

$ apt-cache search haskell | grep network
libghc6-network-dev - Haskell network library for GHC
libghc6-network-doc - Haskell network library for GHC; documentation
libghc6-network-prof - Haskell network library for GHC; profiling libraries
$ sudo apt-get install libghc6-network-dev

Then problem solved, hoped over to ghci and it's there.

$ ghc-pkg field network exposed-modules
exposed-modules: Network Network.BSD Network.Socket.Internal
                 Network.Socket Network.URI
$ ghci
> :t Network.Socket.socket
Network.Socket.socket
  :: Network.Socket.Internal.Family
     -> Network.Socket.SocketType
     -> Network.Socket.ProtocolNumber
     -> IO Network.Socket.Socket
> 

So yeah, if you run into this, there you go. I think everyone looking into this can probably figure it out on their own, but if not, hopefully this helps. Edited the command stuff to just remove some information and lots of text in between.

Thursday, September 20, 2012

Recursive Fibonacci

So after bashing my head against the wall working on Fibonacci recursively and thinking "why on earth is this so slow," I realized the solution was reducing everything to a lot of ones and adding them all up. While addition and basic arithmetic for a CPU is exceptionally fast, this was slow due to the magnitude, and would cause way too much recursion for it to even be considered a correct solution. So I searched the web figuring someone far smarter than myself would have not only realized this, but come up with a real solution for it. Sure enough, I found one. The solution they used was in C, so I found it a bit ugly as it required two function calls (so you can call it with one argument). So I converted it to Haskell.

Before posting, the short explanation is this. Fibonacci numbers are found by adding up the previous two numbers. This is not to say you should call it inappropriately like this.

fib :: Integer-> Integer
fib 0 = 0
fib 1 = 1
fib x = fib (x - 1) + fib (x - 2)

However, this is often the solution I see in tutorials about recursion and very legitimate sources. I feel the reasoning for this is to use only one function call and as little code as possible to make an "elegant", or at least readable solution. The solution I found, while it is not the prettiest thing, it is far more efficient and very fast and after taking some time to really read it, makes far more sense true to the derivation of Fibonacci numbers. So here is the solution I found and converted to Haskell.

fib :: Integer-> Integer
fib 0 = 0
fib n =
    let
        fib2 1 _ n2 = n2
        fib2 x n1 n2 = fib2 (x - 1) n2 (n1 + n2)
    in
        fib2 n 0 1

Credit to the solution here. Glad to see people finding valid recursive solutions and letting people like myself know there's a better way.

Sunday, September 16, 2012

class vs namespace vs struct

I think one of the hardest concepts to get through to newer programmers is what classes are compared to a namespace or structure. More often than not I hear people asking if a class is just a struct with functions. I have also seen an explanation in a structured program claiming namespaces were like classes and object oriented. While all three of these things can on some level emulate one another, I would like to try and clarify the way to correctly use them, as well as how to emulate one another, which is used for languages that do not have such features.

First thing, namespaces and structs are not objects despite other languages accessing them like such. These are methods for structuring programs and data, not creating objects, which is what classes do. So the question may arise, what is an object? Well, the term is almost exactly the same as the plain English word without any context. Objects, excluding static, are individual things to interact with and manipulate attributes. Until you instantiate, or create a new object from a class, you can only interact with static methods. In a language like Java, all the methods of the Math object are static. This means the method does not rely on an instance, meaning it cannot get or alter any attributes of the object itself. In fact, you can use them without making an instance of the class. I believe this is where the confusion arises making people think a class is a struct, only with functions.

When you make a class, the idea is to create a base for an object, like a mold. So let's say we want to make a ball. First we would make a ball class.

class Ball(object):
    def __init__(self, color):
        self.color = color
    def getColor(self):
        return self.color
    def setColor(self, color):
        self.color = color

With this, we can now create a ball of any color we want. Let us make a red ball.

red_ball = Ball("red")

So now we have a red ball. We can change the color of the ball by red_ball.setColor("blue"), and completely throw off the name scheme I did. As such, the idea is we created a ball and now manipulate it through its functions. While it is true we could do this with a struct, the bottom line is we would not have an object, but instead we would have a data structure. Objects also have inheritance and polymorphism. Inheritance would be when you make a class based on another class, thus any functions and attributes from that class are inherited. In my code, I already included some inheritance by making my class based off of object. This allows access to various attributes that would otherwise not be accessible. Here is another class based off of our Ball class.

class BounceyBall(Ball):
    def setBounceLevel(self, level):
        self.bounceLevel = level
    def getBounceLevel(self):
        return self.bounceLevel

Now you may notice I do not declare all the other functions, however they will be accessible. Thus I would create a BounceyBall the same I created a normal ball. There are also further things you can do such as creating wrapper classes and decorators, but those are way out of scope for the basics. Polymorphism on the other hand, it hard to grasp but very simple. It is basically overwriting a method on a class to use the same base class but create different objects. Here's a quick example copy-pasta style because it would take a while to type out my own.

class Animal:
    def __init__(self, name):    # Constructor of the class
        self.name = name
    def talk(self):              # Abstract method, defined by convention only
        raise NotImplementedError("Subclass must implement abstract method")
 
class Cat(Animal):
    def talk(self):
        return 'Meow!'
 
class Dog(Animal):
    def talk(self):
        return 'Woof! Woof!'
 
animals = [Cat('Missy'),
           Dog('Lassie')]
 
for animal in animals:
    print animal.name + ': ' + animal.talk()
 
# prints the following:
#
# Missy: Meow!
# Lassie: Woof! Woof!

There is mention of an abstract method. This is something that is just there because it needs to be, and must be declared in all descendant classes. Python is a bit loose on these, languages like Java have much more enforcement in such things.

So these are all features that separate classes from structs and namespaces. Now what makes a struct so special? Well a struct is a structured data type. Structs give you the ability to create data down to the binary value and call on them with individual names. With a struct you could store 8 flags in a single byte. A struct packs multiple pieces of data into 1 variable. This can include even pointers. While you can do the same with a class, you cannot show this much control over the data itself. Structs can also use function pointers, which can be similar to a class, however it will be a static function.

Finally to namespaces. Namespaces are a way to separate functions and variables, usually in different files, to prevent clashing or redefining something. They are not objects or structs, but code accessed by a name. They can be used like static classes, but not structs. Namespaces can also house classes.

Now how do you decide when to use which? Well, first you see what the language you are using has to offer. Assuming you have all three, here is how you decide. If you are making a piece of code to represent an object, like a character in a game, use a class. If you are creating a grouping of data, use a struct. If you are creating a grouping of functions and possibly variables, you can use either a namespace (preferably) or a static class.

There are many other language specific details, however these are just my thoughts and conclusions on the matter. To fully utilize a language, learn all of its features and what advantages they have so your code will not only work, but will be efficient.

Saturday, September 8, 2012

Recursion

I've been on another functional programming kick, although I have moved over to OCaml for reasons of performance and still having object oriented sides to it. Either way, it allows recursion, which can make life easier on a lot of things. So I've been working on making things with recursion. Easiest place to start was the challenges over on Project Euler. So I looked at the first two challenges and did them.

Now I want to cover some of the problems I ran into so hopefully someone can be helped by it.  So my first problem always seems to be setting up a limit. The reason this can be a problem is sometimes I forget it's easier to work backwards, especially considering in the end recursion is walked through backwards. The limit can be viewed as similar to the conditional for a loop. Without this conditional, it would go on forever. I find limits a bit easier to do in haskell, but for now I will use Ocaml for practice and show a C equivalent. To do an easy one, let's do Fibonacci number recursively.

let rec fib number =
  match number with
      x when x < 1 -> 0
    | 1 -> 1
    | _ -> fib (number - 1) + fib (number - 2)
;;

In this, I laid out two limits, one is to handle should something just act funny, there is a way to do this a little bit shorter if I want to take away that safeguard, however I will leave it there. The first limit is anything less than one will result in a 0 and the second one is that a one will give you a one. The point of a limit is to show where recursion should end and instead give a value. The final piece is for default behavior and will be where the recursion is. The hard part to follow is in the end you get a whole bunch of ones that are then added together as apposed to the actual numbers. In C, it would look like this.

// With recursion
int fib(int number) {
  if (number < 1) return 0;
  else if (number == 1) return 1;
  else return fib(number - 1) + fib(number - 2);
}
// Without recursion
int fib_loop(int number) {
  int x=0, y=1, z=0;
  for(register int i = 1; i <= number; i++) {
    z = x + y;
    x = y;
    y = z;
  }
  return z;
}

With looping, it is slightly longer and perhaps clearer. In situations like this, the solution of which is best is to use whichever fits the language better. The main thing to note different is the variable tracking. Without recursion, variables need to be tracked and handled. With recursion, you can apply a function to the same arguments without needed to track them. This makes code more modular with a stable interface.

Another catch I ran in to was trying to work backwards based on the return type. The thing with OCaml is there is no return statement. When looking at the C version, it becomes much more clear as to what results where. When confused about the return, remember everything falls back to the limits. When it comes to recursion, all limits must return the same type, even if you work in a language that would allow mixed types. Also all recursive functions must return a value.

Now the end advantage of functional programming style is a more modular code, which means you can reuse a lot of stuff and avoid large, specialized functions. Even in the short time I have been reading about OCaml, I have made a couple functions that I liked in other languages that weren't already included and have recycled them to try out other ways of doing things.

Something else with recursion you may notice is that it is simply repeating the same thing over and over until the some limit hits an end point which will result in the answer. A quick way to make something recursively is to break it down into the smallest steps. Then you can divide everything up in a way that it will repeat the same task to get the answer, all without looping. This is not to say looping is pointless, it is just that with recursion you need only design and interface and apply the logic rather than map out variables and track everything. Here are a couple more recursive functions in OCaml.

(* Add up all items in a list of integers *)
let rec sum l =
  match l with
      [] -> 0
    | [last] -> last
    | head::tail -> head + sum tail
;;
(* create a list of integers in a range *)
let range lower upper =
  let rec mk head tail =
    if head < lower then tail
    else mk (pred head) (head::tail)
  in
  mk upper []
;;

Now the first function, sum, actually starts at the very last item in the list and adds it backwards, this is shown in the limit I call "last" which was named to clarify that. An empty list is also in there because an empty list can be passed to it. The second function, range, creates a list, starting with the last element. Here you may also notice I use a recursive function inside the function because we do not use variables. Instead we use the outer function to create the inner function in a sense. Similarly, a lambda style could be used, however is not because this was easier for me.

In my opinion, I think recursion is a very solid method to do a lot of things and wish it was a bit more developed for other languages. However, I am reading about unrestricted recursion and how to apply it with C, so when I manage to wrap my head around that I will hopefully be able to make use of both imperative and functional means in a way I can use the best solution for each case (so hopefully shorter and clearer code).

Wednesday, August 22, 2012

Nested Looping

Nested looping is something of an art. There are many times when I was starting out where I thought a jump statement like a GOTO would make things easier. As I've developed my programming skills, I have learned that GOTO is a waste of time that will create spaghetti code, unreadable and hard to follow. Instead, many solutions can be created with looping. Although sometimes nested looping is needed. Nested looping can be complicated to grasp, but it is actually very simple.

Nested looping can process multi-dimensional arrays, create simple user interfaces, and are commonly used for graphics for staying alive and checking events.

The flow of nested looping is very sequential. For every time to outer loop executes, the inner loop will execute however many times it is set to loop. So say you have two nested loops to populate a 2 dimensional array where it is an array of 10 arrays of 10 integers, the inner loop will run 100 times.

int val = 0
for(int i = 0; i < 10; i++) {
    for(int z = 0; z < 10; z++) {
        array[i][z] = val++;
    }
}

This will populate the array sequentially, starting with the first array. It will go through that then move on to the next array, so on and so fourth. Nested looping does not always have to include just loops and does not always need to be iterative. Here is some psuedocode showing a very basic user interface.

while(keepGoing) {
    refresh();
    events[] = event.get();
    for(event in events) {
        if(event = QUIT) keepGoing = 0;
        else ;// Some event handlers
    }
}

It is crude, but I didn't feel like mocking up a full featured one. On top of the extra code, you can also nest multiple loops on the same level or nest on even more levels. Perhaps it would make a nice way to mark up the flow for something to solve a sudoku puzzle or something like that.

However, sometimes there is no clear and linear way to process information. Or perhaps there is but it is messy. You may want to use a GOTO. Before you do, consider recursion. By using multiple functions to achieve a task, your code will be more scalable, compartmentalized, cleaner and shorter for these dire situations. If the recursion will be very deep, I suggest looking into lazy evaluation and if you can use it.

Just some simple recursion and branching can allow you to jump to any point in the code without complicated looping and tracking of variables. The idea is that when you enter a function, you then branch out to other functions selectively, allowing you do go in a non-linear flow. One important caveat, make sure a limit is in place so the recursion will actually end. If it does not, you will either get an error or it will run forever depending on how you implement it and in what language.

Sunday, August 19, 2012

Multidimensional Arrays

Multidimensional arrays are hard to comprehend at first, but they are actually very simple. The problem arises in people using multidimensional lists and similar data structures that can make things awkward as they do not function the same.

In my previous post on arrays, I mention that an array will contain all the same datatype. The reason for this that I was trying to get across is because since an array would be contiguous memory, to find any element you simply multiply what element by the size of the datatype (since all are the same datatype), which will result in how many bytes to add to the memory address to get the address of the element you wish to access. It would be like arranging items an equidistant apart and telling someone to find an item where instead of counting, you use a ruler and measure out where the start of the index is to get the item.

Since an array requires all datatypes to be the same to easily find the item, multidimensional arrays work the same. A multidimensional array is just an array of arrays, and as such all arrays inside the array must be the exact same length, meaning also they are the exact same datatype. If the items are not the same length, a couple things can happen. The first thing is the memory will still be allocated, but the unused elements will be empty. This will preserve the speed to access the information and in some instances is called "padding". Padding is filling up unused space with empty information so that it will reach a certain size. It is used in some database designs and will be used for an array. Another thing to consider is an array of strings. Strings are arrays themselves, more specifically character arrays. In the case of an array of strings, all strings will be padded to the longest string in the array. The end of a string is determined by the null character, denoted as \0, which is character zero (0x00). This also means a string is one character longer than what you normally see, as programming environments like compilers and interpreters deal with that for you.

So the math to access an element in a multidimensional array pretty much the same as a single dimensional array, except you also tack on the length of any preceding arrays. Like if you have an array char[10][10], the first index is 10 characters in length for each index, then you add on the last part normally.

(sizeof array) * i1 + (sizeof type) * i2

This is the basics of a two dimensional array and can be expanded for nth dimensional arrays.

Friday, August 10, 2012

Looping

After polling for an idea for a post, one on looping was asked for, so hopefully I can shed some light on looping. First thing to note is that not every language loops the same. The two most common loops I have seen are for and while. There are also foreach, do...while, and various syntaxes that are similar to or emulate these ideas. You can also "loop" with recursion, which is something more for functional programming. So for now we sill stick to the common loops and I will cover the syntaxes I know for certain languages.

First off, there are two kinds of looping. Iterative and conditional, the first being what most people think of when looping. Iterative looping requires going for a fixed amount of times and commonly used for processing data iteratively, or one element at a time. Conditional looping will most often, though not always, use an infinite loop and some condition to break out of the loop. This is what you would use for a network application or anything where it is unknown how many times you will loop. To start, let us look at iterative looping.

Iterative looping will often use a for or foreach style syntax. Since foreach is not very common, I will stick to the C style for for now. This type of for loop uses 3 optional statements for the condition, variable for tracking, condition, and an update. The common layout is

for(int i = 0; i < dataSetLength; i++)

In this, i is usually the index of the loop, dataSetLength is the amount of times to loop, either the length of a set of data like an array or just an arbitrary number to preform a task and i++ will increment the index for each time it loops. The main use of this is for a linear style processing, like so

int x[10] = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9};
for(int i = 0; i < 10; i++) x[i] += 2;

Now foreach style looping is much easier to use than all this extra variable manipulation, however each language seems to have its own syntax for this, so I will breeze through these. C does not have one, so you would either need to make a macro for it or use the standard for loop.

Python, one of my favorites, uses for variable in iterable, where iterable is something like a list or dictionary. Simply it looks like

x = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
for y in range(len(x)): x[y]+=1
This for variable in dataset style syntax is also in Ruby and Javascript, but with parenthesis around the condition part in Javascript, (variable in dataset).

Java has its own syntax which was borrowed by C++, which is simply
for(type variable: dataset)

The last one I'll look at is php's foreach loop, which looks a lot like Perl's.

foreach ($iterable as $item)

All of these can be made with a standard for loop, save for in the languages that do not use a standard for loop, like Python, a while loop or generator expression can be used for a standard for loop flow, as the for is a foreach. To use a while loop like a for, in C syntax it is

int i = 0;
while(i < dataSetLength) {
    // Do stuff
    i++;
}

I'll leave converting it to other languages as while loops to yourself since it is pretty standard throughout many languages. Now we turn our heads to conditional looping. Conditional looping, as previously stated, will take an infinite loop (not always) and loop until a condition terminates it. This can use while and do...while loops or for loops in some languages, however a while loop is generally preferred for clarity. There are two C style infinite loops.

while(1)

and

for(;;)

Infinite loops while use a break statement in a conditional to terminate the loop. This would be used for something like a user interface that collects data indefinitely or a network app that needs to keep getting and sending information until some arbitrary point or anything else where you are unsure of how long it should loop for. if a single condition exists that terminates the loop, one could use a variable and do something like

while(condition)

or

for(;condition;)

For this, a while loops would generally be used. Instead of checking a condition, you would even simply use just a variable name assuming the variable can be tested as either true or false, which is also easy and in some cases clearer. I suppose the easiest way to understand this is to show an example. So we shall make a simple cli.

#include <stdio.h>
#include <curses.h>

int main(void) {
    initscr(); // Initalise the ncurses library
    cbreak(); // option to disable buffering etc.
    char x;
    while (1) {
        printw("0 to exit;> ");
        refresh(); // update the screen, so the text is displayed
        x = getch();
        if(x == '0') break;
        printw("\nBlah blah blah\n");
        refresh();
    }
    getch();
    endwin(); // close ncurses library
}

While this program doesn't do anything really, it gives an example of how to use an infinite loop. As you should notice, the break happens in the middle of the loop, so if the condition is met it will break out before it executes everything in the loop. If we moved the condition in the if statement to the while itself, it will not check the condition until everything in the loop executes. This is why jump statements are necessary. Here is a more complex example using another jump statement.

#include <stdio.h>
#include <curses.h>

int main(void) {
    initscr(); // Initalise the ncurses library
    cbreak(); // option to disable buffering etc.
    char x;
    while (x != '2') {
        printw("0 break; 1 continue; 2 end> ");
        refresh(); // update the screen, so the text is displayed
        x = getch();
        if(x == '0') break;
        else if (x == '1') continue;
        printw("\nBlah blah blah\n");
        refresh();
    }
    getch();
    endwin(); // close ncurses library
}

In this case, 0 will exit the loop, 1 will skip sending text and 2 will show the text then end. Now there is one more alternative, a do...while loop. These loops are the same as a while loop, however guaranteed to run the code in the loop at least once (excluding if a jump statement is met). With some minor tweaking, we could utilize it in a useful way.

#include <stdio.h>
#include <curses.h>

int main(void) {
    initscr(); // Initalise the ncurses library
    cbreak(); // option to disable buffering etc.
    char x;
    do {
        printw("0 break; 1 continue; 2 loop again> ");
        x = getch();
        if(x == '0') break;
        else if (x == '1') continue;
        printw("Blah blah blah");
    } while (x == '2');
    getch();
    endwin(); // close ncurses library
}

This will make it so that 2 will keep the program looping. The reason we need the do...while instead of a normal while is because without that, the program would not execute.

This is just a brief overview of how to use some loops. Sometime later I may cover nested looping. The summary is looping will repeat code until a condition is met. That condition can either be a set amount of time or indefinite, it all depends what you need to do. Something to always keep in mind is how many times a program will loop and how fast. Looping with some blocking statement, usually i/o, will prevent it from using too much cpu by running code as fast as possible, even if not doing anything. As such, if using something that is not blocking, meaning it will not stop and wait for something else to happen, a frame rate of some kind is suggested. I will cover timing in another post, probably with nested looping.

Wednesday, August 8, 2012

Understanding Arrays

I was out of state for a bit and a friend I wanted to asked me for some help with his programming work for school. Now being as I'm used to Python, I did not understand fully what the goal was but I feel arrays are something people need to understand. So I am going to explain the basic concept of an array in C terms and hopefully that may clear up some confusion. Keep in mind my explanation is not 100% true and I will explain the catches when I get there.

An array is a piece of contiguous memory allocated to store n number of elements of the same data type. In essence, it is a variable. However technically it is a pointer. A pointer is a variable that contains an address to a particular memory address. For example, if you have

int arr[10];

Here we have an array of 10 integers. The variable arr is not the array, but it points to the memory location at the start of the array. Thus arr is not the same as arr[0], as this accesses the first value. In a more diagram-like example you have

arr => [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
        ^
        arr[0]

Now say we want to get the second element (arr[1]). To do this we then need to start at where arr points to, then move along that memory point to the next int. To get to this we need the memory address arr then to add the size of an int to it to get the next value. Thus to get this we can do

((sizeof int) * n) + arr

Where n is the array index arr[n], in this case 1, so we end up with

((sizeof int) * 1) + arr

This will give us the memory location of the second element, 0 being the first because arr points to the start of the array, but not the first element. That is the basic idea behind an array where the syntax given is more or less an alias that goes all the math automatically for you.

So now for some caveats. Higher level languages may use memory abstraction. This means the memory in reality is not contiguous, however to the program it appears as it is. The operating system further abstracts this with methods like paging, which allow for more efficient memory usage at the cost of some overhead for an abstraction process. To the program, the memory will appear contiguous to the program. Due to memory abstraction methods, you should not rely on contiguous memory and also use bounds checking where necessary as apposed to waiting for a segault or some other thing to hit the fan.

Uses for arrays include tracking sets of data and easy looping, accessing and assigning of data, for linear or non-linear processing, whereas lists are more suited for linear searching. With lists you can also create trees, which will allow different ways to access data. Arrays can also be used for sorting data and for a lot of things will be a good deal faster than lists and trees, but in some situation can be cumbersome.

Array manipulation is something I hate, like permutations and sorting. Easiest options are using an array manipulation class, as doing things manually is a pain and will need temporary variables, unless you can efficiently do recursion like with a lazy evaluation language (functional programming languages do this and are very clean with these methods because of it). So  before attempting something like that on your own, check for a library that can do it for you. If you absolutely need to do it yourself or just want to for knowledge seeking, good luck.

Sunday, July 29, 2012

Some Tech Humor

Another not-so-technical post. I've come across a lot of rather funny tech-related comments and web pages and there are some that I find very clever, specifically jokes made with code. So I will share some of them.

Some stuff in C
// Sometimes I feel like the compiler ignores my comments

#include <stdlib.h>
#define TRUE random()%2

int random(void) {
    return 4;
}

Malloc everything, free nothing!

COBOL
PERFORM SEX WHILE MAN EQUAL TO TASK

REPEAT
 ...
UNTIL THE-COWS-COME-HOME

Java
Exception up = new Exception("Something is really wrong.");
throw up;

} catch (PartInitException pie) {
// Mmm... pie
}

 try {
 //...
 } catch (SecurityException sex) {

Miscellaneous
double penetration;

Disallow: /harming/humans
Disallow: /ignoring/human/orders
Disallow: /harm/to/self

If I come across anything else that I find funny, I'll add it here.

Friday, July 27, 2012

Technical Speak (some thoughts and advice)

This post is not going to be technical so much as I really want to write about some tech things people say that are either completely wrong or just down right annoying. So take this how you will, but this is an opinion piece.

Something I once did before I knew much about computers but now that drives me crazy when I hear it, is when people talk about Java and call it Javascript or visa versa. Let us get a few things straight. Java is not related to Javascript nor the other way around. Javascript is a scripting language based on ECMAScript whereas Java is an interpreted language developed by Sun Microsystems. While Java can be used as a web technology, it is not the same as Javascript and while they have some similar naming conventions, any real depth into some code would reveal they are very dissimilar. So stop mixing them up, it is frustrating.

Another irritating point for me was created by the media, "Mac vs PC." The real goal of this is Mac OS vs Windows. Mac computers are PC's, as PC stands for personal computer. Furthermore, Mac computers can run Windows and if you're crafty, you can run a Mac OS on another PC, virtual machines being the easiest to get things going. Also some people throw Linux into the list. Well, OS X is Unix based as well. So they have some similar things to them. As to which is better, all three of them suck. It is just a matter of picking which one sucks less at what you personally need to accomplish. None of them are good at everything no matter what they claim.

Overuse of acronyms. In the technical world, many acronyms exist. Many acronyms overlap and it depends on what you are talking about. Overuse of acronyms in situations where it is not necessary is annoying and even a tech geek can get lost in such conversations. Granted there are exceptions where some technologies are explicitly referred to their acronym based name because some of the names are long and annoying. Some examples are XML (eXtensible Markup Language), HTML (HyperText Markup Language), or even php (Personal Home Page/Hypertext Preprocessor [it has changed over time]). Choosing to use all those acronyms in an environment outside of a situation that deals directly with such things is annoying, confusing and not many people can follow clearly. Use words.

Being able to navigate a website does not make you tech savvy, same goes for navigating software. I am relatively tech savvy and I get lost on so many websites and various user interfaces. One annoying example is of Microsoft Office. They keep changing the interface, and while I know what I want to do, I can spend a lot of time looking for various options. These things are esoteric and not standardized. While many try to keep similar conventions in an attempt to create better ease of use, they do not have to. Reliance on people trying to make similar and familiar interfaces is not possible.

Do not use tech terms you do not understand. If the media is to teach us anything, let it be this. For example the famous CIS line, "I'll make a GUI interface in visual basic, see if I can track an IP address" (GUI pronounced "gooey"). Taking this I would have to respond, okay, I will make an OS in Piet, see if I can make a super computer. Despite the lack of need to sum up why this is dumb, I am going to do it anyway. A GUI is not necessary for any of this. A good deal of tools for sniffing are done in command line interfaces because there is no point in the extra computational time and effort going to making a pretty window. Visual Basic is a dumb choice. I don't even know if you could make a sniffer from VB. If you can, it is probably some random .NET feature that is a waste of time. Now you do not "track" an ip, you can trace a route. Tracing would be following something along its path. That is a small thing to pick at, but tracking an ip would be like sniffing just that ip. Tracing would be finding the origin.

Using tech terms from one thing for a separate case.When drawing a comparison it is fine. However, using that comparison like the cases are equal is usually not fine.

Comparing things on levels that they are not comparable. I am sure at some point you have come across things like Perl vs Python, where they argue about dumb things like which is easier to read or application. Seeing as how Perl was more so made for text processing and Python is more of a general purpose language, they are obviously not going to be good at the same things. Benchmarking comparisons are one thing, but arguing about readability or which way of doing things is better is dumb because it depends on the application and the person.

Skids. Script kiddies. People that claim to hack when they are using premade tools without knowledge of the underlying innerworks of how or why things work they way they do. Downloading things like custom firmware to break a game system or opening up a tool to do some network scanning. Some of these can be dangerous if you do not have some understanding of how they work. In this light, the use is not that of a tool but a black box effect. Someone puts something in and gets the desired result out without understanding what just happened. So not everyone that uses these tools are skids or something to that effect, some actually understand in general how they work and use a premade tool to build something else faster.

So those are just some thoughts and opinions based off of things I have heard people said that just really catch me the wrong way and I have seen them criticized for saying such things as well. An easy way to keep yourself from saying something that will annoy others, whether you are right or wrong, is to remember you do not need to prove anything to anyone what you know, because you already know you know it.

Sunday, July 22, 2012

IRC Bot (logbot)

After my first post on IRC bots, I decided to make a bot for logging a chat that was a bit better designed than the one I posted. It is far from perfect, however it will write formatted logs to a file for you, won't sit around if it is kicked and if it sees a message it cannot format, it will log the raw data. Also to keep in mind, it only joins a channel ofter the motd finishes with a certain number code. If there is no motd or it does not send the right number code it won't join. The reason for this is I just wrote the thing today and don't have the time to check all possibilities. So use it as a skeleton and develop on it further. Or ignore it and move on with your day. All descriptions will be commented in the code.

#!/usr/bin/env python
# Import all the packages we will be using
import logging
import socket
import ssl
import os
import sys
import re

# Some global variables for configuration
HOST = "irc.69megabytes.com"
PORT = 6697
NICK = "PyLogBot46"
CHANNEL = "#allthefallen"
SSL = True
FLOG = os.path.join(sys.path[0], "ircbot.log")
PWD = "supersecretpassword"
# This is for use later on to know when to join the channel and do it only once
notInChan = True

# Regex parsing of a standard IRC message
def parse(data):
    temp=re.match(r"^(:(?P<prefix>\S+) )?(?P<command>\S+)( (?!:)(?P<params>.+?))?( :(?P<trail>.+))?$", data)
    if temp:
        temp=temp.groupdict()
    return temp

# Where the magic happens
if __name__ == '__main__':
    # Set up the logging format
    logging.basicConfig(level=logging.DEBUG, format='%(asctime)s %(message)s', datefmt="%Y-%d-%m (%H:%M:%S %Z)", filename=FLOG)
    # Message format strings so we can easily alter them
    format_privmsg = "{1} <{0}>: {2}"
    format_notice = "(notice) <{0}>: {1}"
    format_join = "{0} has entered {1}"
    format_nick = "{0} is now known as {1}"
    format_mode = "{0} set mode {1}"
    format_modes = "Mode set {0}"
    format_kick = "{0} kicked from {1} by {2} ({3})"
    format_srvmsg = "-!- {0} {1} {2}"
    try:
        # Create socket
        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        # Set up SSL if necessary
        if SSL:
            s = ssl.wrap_socket(s)

        # Connect to the server
        s.connect((HOST, PORT))

        # Authenticate
        s.send("NICK " + NICK + "\r\n")
        s.send("USER " + NICK + " www.anarchy46.net PY :Python LogBot\r\n")

        # Main loop
        while 1:
            # Reset information received and processed
            data = ""
            info = {}
            # Retreive data 1 byte at a time, stop at newline
            while data.find("\n") == -1:
                data += s.recv(1)
            # Remove unnecessary whitespace
            data = data.strip()
            # Parse the raw data
            info = parse(data)
            # Privmsg handler
            if info['command'] == "PRIVMSG":
                logging.info(format_privmsg.format(info['prefix'], info['params'], info['trail']))
            # Notices
            elif info['command'] == "NOTICE":
                # If to self from someone
                if info['params'] == NICK:
                    # To quit properly
                    if info['trail'] == PWD:
                        s.send("QUIT :Done logging.\r\n")
                        break
                    # So we don't log the password
                    logging.info(format_notice.format(info['prefix'], info['trail']))
                # Server notices
                else:
                    logging.info(format_srvmsg.format(info['prefix'], info['params'], info['trail']))
            # Respond to server pings (no need to log)
            elif info['command'] == "PING":
                s.send("PONG " + info['trail'] + "\r\n")
            # Someone joins
            elif info['command'] == "JOIN":
                logging.info(format_join.format(info['prefix'], info['trail']))
            # Someone changes nick
            elif info['command'] == "NICK":
                logging.info(format_join.format(info['prefix'], info['trail']))
            # Modes
            elif info['command'] == "MODE":
                # Normal mode changes
                if info['trail']:
                    logging.info(format_modes.format(info['trail']))
                # Modes for self (set by server)
                else:
                    logging.info(format_mode.format(info['prefix'], info['params']))
            # Someone gets kicked
            elif info['command'] == "KICK":
                # Quit if it's the bot
                temp = info['params'].split(' ')
                if temp[1] == NICK:
                    logging.info("Kicked from channel.")
                    s.send("QUIT :Bye bye...\r\n")
                    break
                # Log everyone else being kicked :D
                else:
                    logging.info(format_kick.format(temp[0], temp[1], info['prefix'], info['trail'])
            # Special server messages
            elif info['command'].isdigit():
                if info['trail']:
                    logging.info(format_srvmsg.format(info['prefix'], info['command'], info['trail']))
                else:
                    logging.info(format_srvmsg.format(info['prefix'], info['command'], info['params']))
                # End of MOTD
                if notInChan and info['command'] == "376":
                    s.send("JOIN " + CHANNEL + "\r\n")
                    notInChan = False
            else:
                logging.info(data)
    # Abort!
    except socket.error as err:
        logging.critical("Socket error: " + str(err.errno))
        sys.exit(err.errno)

It's not the prettiest of code, but it works. When launching from the command line, make sure to add an ampersand (&) at the end so you don't need to wait for it or leave a window open with it running. Enjoy, and remember this thing only took me a couple of hours to write and sort out some formats.

Saturday, July 21, 2012

Making an IRC Bot

A common question I get from people I know is about making an IRC bot. The problem I run into with people trying to do this is they are usually very new to programming and half the time they are struggling with the basics of simply how to write a program. So before getting into more detail, there are some prerequisites you need to know first.
  • Understanding the programming language syntax you want to use
  • Basic programming
  • Networking knowledge of basic sockets (nothing in depth but more so how network communication should be modeled)
  • Knowledge of the IRC protocol raw (access to an RFC is very useful)

Now f you are still reading, I will assume you at the very least plan on doing the research before actually trying anything. For this, I will be writing it in Python as it is very simple and easy to do. So let us work through this piece by piece.

The first thing we want to do is run our imports so we can have all the modules we need to work with. Rather than using some third party IRC module, we will work with raw sockets because I can and it gives more insight as to what is going on. We need the module for sockets, and regular expressions. We will also import time and I'll get to why later.

import socket
impor re
import time

So now that we have the packages we need, now we need to create a socket object. With a socket object, we can establish a connection over TCP/IP, among other types of socket connections. For our purposes we will use IPv4 and a stream. For this we use the variables AF_INET and SOCK_STREAM. The AF in AF_INET stands for "address family." Now let's create the socket.

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

Now we have the socket object s. In another post I will explain how to put an SSL wrapper on it to use a secure connection. As for now, just a socket will be used. Now we need to connect to a server.

s.connect((HOST, PORT))

Here the connection info is passed as a tuple, so yes the parentheses are on purpose. In this, HOST will be a string of either the IP address or domain and PORT is the port number you will connect to. Now that the connection is established, we need to send identifying information for the IRC. To simplify things we will have the program sleep for a second as some servers do not let you authenticate immediately and instead send a signal for it. However I do not feel like writing out something to check for that signal right away.

time.sleep(1>)
s.send("NICK MyIRCbotNick\r\n")
s.send("USER MyIRCbotNick anarchy46.net PY :Python Bot\r\n")

Okay, so now that we've authenticated, we will sleep again then join a channel because yet again, cannot join right away.

time.sleep(1)
s.send("JOIN #somechannel\r\n")

Now we set up the loop to read the server data. Because this will go on indefinitely, we will use an infinite loop.

while 1:
    data = "" # Reset data each time
    while data.find("\n") == -1:
        data += s.recv(1) # Receives data 1 byte at a time
    data = data.strip() # No need for leading or trailing whitespace
    print data
    # Watch for and respond to server pings
    if re.search(r"^PING", data):
        s.send("PONG" + data[4:])

Now you have all the code needed to connect to an IRC server, join a channel and stay connected. A few things to keep in mind. The socket is blocking, so the bot will wait on the recv() until it gets some data. Without blocking the program would be too CPU intensive without some extra work. Also note that printing data will print raw IRC data, which is a lot different than the messages you see in a client, and will include control characters like char 1 and 3. Furthermore, there is no error handling or autojoining should the bot be kicked. This is simply a small and poorly made IRC bot that is to demonstrate what is needed to connect to an IRC and I do not recommend this design as a base for a larger scale bot that will be used more often. Sometime later I will divulge a more robust bot that has the purpose of logging an IRC channel or channels.

Friday, July 20, 2012

Generator Expressions vs. List Comprhension

Something I seemed to have always overlooked in Python was generator expressions. A generator expression uses almost the exact same syntax as list comprehension, however instead of square braces [] it uses parentheses (). The difference here is that a list comprehension will create an entire list and put that in memory. A generator expression creates a generator object, which will only give the value when it is needed. So the advantage in a generator is that it will use less resources and still be able to have values like a list. This makes them good for when you need to evaluate a list of items one at a time as you don't need to load a whole list of items up just to process one data value.

After some reading, there are also some improvements that will eventually be moved to list comprehension. One of these is variables in list comprehension are in a global scope where as with generators they are not. This will prevent side effects (altering a variable by accident). You can also pass a generator expression to a function without the need for the parentheses around it as the function parentheses count. However if there is more than one argument needed, it will need the parentheses.

It should also be noted that a generator expression returns a generator object. List comprehension returns a list. This means for functions that manipulate or require an entire list will not work with generator expressions. A quick example would be random.shuffle().

So the idea of when to choose which one to use would be based on knowing when or how you will need to process the values. For single value processing, a generator expression is ideal, whereas list comprehensions are more for when you need the whole list there and ready to go, possibly with a need to manipulate the list itself. You can also use a generator expression to create a list like list comprehension bu placing it in a list(), which won't increase performance.

Sunday, July 15, 2012

Python lambda (some uses)

I previously brushed over a bit about how lambda in Python works. With it, you can achieve a more functional style design. Using lambda, you can pass a function as an argument without defining another function and then passing that function's name. One thing I do not think I fully clarified on is that lambda is functional programming and in functional programming it must always return a value. As such, an if/else is not used in lambda as it evaluates one statement and returns the result. This can be useful for a shorter and cleaner way to use certain built-in functions, primarily the ones that effect lists. They include functions like map, filter and reduce. Functions like these will iterate through a list and apply the function to the items, either making a new list or returning a value based on what was applied. This can be achieved with list comprehension as well, but not for all of them.

So first let us look at the map function. This will go through a list passing the item to the function and then places the return value in the list it will return. So let us take a simple example of getting the second power of a list of numbers without the map function.
def square(thelist):
    returnList = []
    for item in thelist:
        returnList.append(item**2)
    return returnList

As you can see, that is a very specific function and long to write out for something so simple. Now we shall turn that into a single line with a single function call using the map function.
map(lambda a: a**2, theList)

This is much smaller and simpler to use, as well as clear to anyone that knows lambda is a function. We also did not need to write out and define a function, thus reducing something simple to reflect how simple it really is. Now to convolute things a bit with some list comprehension.
[item**2 for item in theList]

In another post I will explain how to pick the most efficient way for doing the desired task, but for now I will stick with the uses of lambda and equivalent ways to write the same thing.

Now let us look at the filter function. What this will do is similar to map, only it checks for a true or false (or an equivalent that will make a true or false) and return the original value. Sticking with a math feel, let us get a bit more complicated and check for every perfect right triangle, perfect being one that has all integers for side, like 3, 4, 5. To do this, we will use the Pythagorean theorem (a2 + b2 = c2), avoiding using the square root as that tends to be more computational intensive.
def perfectTriangle(thelist):
    returnList = []
    for item in thelist:
        a, b, c = item
        if a**2 + b**2 == c**2:
            returnList.append(item**2)
    return returnList

In this function, it requires the list to be in a tuple of 3 items as the sides of a triangle, the c is always the hypotenuse. If you don't know this math stuff, go do yourself some reading. Now let us apply this to the filter function.
filter(lambda (a,b,c): a**2 + b**2 == c**2, theList)

With the lambda function, there is no if required. This is because it is returning a boolean value which will suffice the if/else style. And now the list comprehension version.
[(a, b, c) for a, b, c in theList if a**2 + b**2 == c**2]

Simple enough. In all of these there is a check to see if the value is True or a value that makes an if statement true and return a list where only the values evaluate true for the condition. Easier to understand the code than the worded version I suppose.

Now finally we look at reduce. This one is different from the others because it does not return a list. It "reduces" the list, or condenses the list, based on a condition you give it. Also unlike the others, 2 arguments are used here instead of the original single argument we have been using (of course making a list of tuples like in filter allowed for a multiple argument style while still only using a single list item). For this one, we shall multiply every number together in the list, giving us something similar to a factorial result.
def factorial(theList):
    retValue = theList[0] * theList[1]
    for item in theList[2:]:
        retValue = retValue * item
    return retValue

As you can see, no list is returned. However doing something like this is a pain and if there is no built-in method or optimized module function to use, we can make a cleaner looking version with the reduce function.
reduce(lambda a, b: a*b, theList)

Nice and sweet. There is also the option adding an initializer argument, in that case instead of starting with the first 2 items in the list, it will start with the initializer and the first item as arguments, respectively.

These are just some of the uses for lambda and built-in functions that they can make easier to use. It is best to use them for a functional style of programming and they can be used in functions you define. To do so, you simply call an argument in your function. Here is a quick example.
def transform(f, data):
    return f(data)

This is a useless function that will take a function and an argument, apply the function to the argument and return the end result. You can also make a function generate a function and return that.
def makeFunction(modifier):
    return lambda data: data + modifier

You can achieve similar results with nesting functions. The piratical application of such things is up to your imagination. Through these, there also lies the possibility that coupled with python decorators you can emulate to some level lisp macros should you be creative enough to do so, however limited. Hope this helps someone understand lambda functions and their uses a bit better and once I create a decent enough application of using lambda to a larger extent, I shall post the code. Until then, happy coding.

Saturday, July 14, 2012

Flip a web page

I know there are a lot of people out there that like messing with others, so I got bored and decided to make a quick little bit of Javascript to do that. This will only work in browsers that support CSS3 and is a bit lengthy because certain browsers need their own special tags. So if you want to flip a page someone is browsing, or maybe even yourself, just enter this code in the address bar of the browser to flip a site:

javascript:document.body.setAttribute("style", "transform:rotate(180deg);-ms-transform:rotate(180deg);-moz-transform:rotate(180deg);-webkit-transform:rotate(180deg);-o-transform:rotate(180deg);");void(0);

It's as simple as that, the page should be flipped. Also, here's a link to do it, you can set it as a bookmark to make a bookmarklet: flip

Friday, June 29, 2012

Binary Tree Search (Haskell)

As of recently I've been looking into Haskell for a chance to better understand functional programming. I'm finding the logic set used for such a style really helps in breaking down a task into smaller tasks, as apposed to always viewing things as a whole and breaking them down into steps. Hard to understand, but it makes sense to me. Anyway, I wanted to challenge myself to see if I was really learning, and decided why not make a searching algorithm. I've never made one before, so I picked one with a concept I understood and already knew the logic of. So the task really was applying these steps in a functional way.

So the basics of a binary tree is you check the middle of an ordered list, if it matches, it's true, if it's greater than that, you check the middle of the first half, if it's less than that, check the middle of the trailing half, rinse and repeat until either all options are exhausted or you find the value. Anyway, here's my code for it.
bt :: (Ord a) => a -> [a] -> Bool
bt _ [] = False
bt x xs
    | x < item = bt x $ take itemNum xs
    | x > item = bt x $ drop (itemNum + 1) xs
    | otherwise = True
    where
        itemNum = (length xs) `div` 2
        item    = xs !! itemNum

On a comment to Haskell, I'm not finding that the language is elegant but instead that functional programming itself it elegant and solid in build compared to all the tracking involved in imperative programming. Haskell itself is a bit cryptic and seems to lean far away from syntaxes I'm used to, but I am slowly getting accustomed to. Hopefully in the future I can have some decent searching and sorting algorithms, maybe even ones for solving puzzles or path finding. Should be fun.

Thursday, June 28, 2012

MyBB Registered-only view BBCode

Decided to make another quick MyBB addon, this adds some BBCode. You can use [paranoid][/paranoid] tags to prevent non-registered and banned members from viewing stuff selectively. I am using it to cover up various resource links. If you are logged in, it will appear like nothing is changed, you can only tell what's covered up by logging out or being banned.

[Download v1.0]

Maintained: https://github.com/anarchywolf46/MyBB-Plugins

Friday, June 22, 2012

Firebug Quick Info (remove)

One of the things that has been annoying me on firebug, is the quick info popup. It's always in the way. So I looked up how to disable it because I didn't have a clue where the options where. In my version of firefox, under Tools > Web Development > Firebug > Options and clicked to uncheck the "Quick Info" option. I think on older versions it was just under the Tools menu. Either way, if you didn't know how to do that, now you know.

Friday, June 15, 2012

Python Lambda Intro (with simple example)

While really a simple thing to do, I wanted to post an example of a very simple program using something a little less common. Instead of the more common way of making a function, I will instead use lambda. In Python, lambda is used for passing an anonymous functions to a method or can be used to have a function generate simple functions. For the latter, decorators can be used for a more complex function generation, however I have yet to fully grasp using it, so I'll save explaining that for when I know how to practically use it.

Now anonymous functions are cool because you can either pass them to a function without creating some permanent function that will only be used in that one particular instance or you can pass it to a variable reference. Passing to variable reference is as simple as defining any variable. The difference in this particular instance is that lambda is limited to very simple functions, and returning variables is not as clear as a return statement. Let's view a quick example.

import sys

ftoc = lambda(f): (f-32.0)*(5.0/9.0)
ctof = lambda(c): (9.0/5.0)*c+32.0

while(1):
    try:
        print """Pick a conversion (EOF to end ctrl+z win/ctrl+c *nix)
1 - Fahrenheit to Celsius
2 - Celsius to Fahrenheit
3 - Exit

>""",
        choice = raw_input()

        if choice == 3:
            break;
        else:
            print "What is the tempurature?",
            temp = raw_input()
            if choice == "1":
                print str(ftoc(float(temp)))+" Celsius"
            elif choice == "2":
                print str(ctof(float(temp)))+" Fahrenheit"
            else:
                print "Unknown option"

    except KeyboardInterrupt:
        break;

    except Exception as e:
        print "An error occured: "+str(e)
print "Goodbye."
exit(0)


The main point to focus on is the 2 lines following the import statements. To break it down, ftoc = lambda(f): (f-32.0)*(5.0/9.0) is actually very simple. It starts out with a normal variable declaration, variable name being ctof. We then assign it to the value of the lambda statement. The word lambda in Python means an anonymous function, then the (x) is the arguments the function accepts. This does not need to be in parenthesis can could just as easily be lambda x:. I use the parentheses to more clearly show that it's not the function name, but arguments being passed to it, I find it to be more clear for myself reading it. The next part is just a formula for converting the temperature. The question may arise, where is the return value? Quite simply, it's whatever the code in the lambda statement evaluates to at the very end.

Now some caveats. Lambda functions cannot span multiple lines, it's only a 1 liner function (with the exception being if a docstring is used). It also supports nested scope, so if it is within a scope where it uses a variable name, it will use that variable, however if the variable is the name of an argument variable, that value will be used instead. Finally, semicolons are ending delimiters and cannot be used to include multiple lines of code, it is only a single statement.

One more little catch I ran into, while I doubt anyone has a need for it, trying to define a list of lambda functions through list comprehension will not work to make different functions. Using something like y = [lambda (x): x+z for z in range(10)] will result in all 10 list items having z as 9 because it will use z as a reference to z and not replace the value, therefore the last value of z is the one that is used. However, if you replace the lambda function with a function that returns lambda functions, it works, and looks like this.

def a(x):
    return lambda(y): x+y

y = [a(z) for z in range(10)]


I was also informed of an alternative way to do this through default values, which looks a little uglier, but can condense things if you really want to. You can do something like y = [(lambda x, z=z: x+z) for z in range(10)], which will place the value of z as the default argument. Note however, that since it is a default argument, it can then be changed, which does not seem like a clean solution.

Wednesday, June 13, 2012

Javascript "use strict";

I came across this in some generated Javascript and got curious as to what it meant exactly, so I looked it up. I've also spent some time reading the notes on it over at MDN, and I'm going to try and summarize it from what I could understand a bit shorter.

Now first thing, this is a new standard for ECMAScript5 (what Javascript is based on, as well as other stuff like Actionscript). It should look familiar to Perl programmers, as Perl has a use strict; which both have similar goals. Now the thing with how it was implemented in the ES5 standard is that if what runs it does not support strict mode, it won't throw an error or warning because it does not add any new syntax. So if your browser doesn't support strict mode, it won't interfere with anything. This could also lead to my own theorized problems in the future, but I'll get to that later.

First, how to use strict mode. You can either make a function in strict mode by placing "use strict"; in the function or make everything strict by placing that at the top of your script.

So what does strict mode do? Makes Javascript throw more errors. Sounds bad, but this is good for those that like clean code without useless crap left behind that could lead to bugs. What do I mean? Well, things that would normally go by silently and either do some unexpected behavior or nothing at all, will throw an error. The best example I see from this is when using a variable. Normally if you do something like

function a() {
    something = 0;
}

Then something will be a global variable because it was referenced before being declared. Now you need to declare a variable before using it with var. This can prevent problems of not paying attention to scope and accidentally overwriting something or if you typo and accidentally make a new variable.

Now there are some variables you cannot overwrite, most for obvious reasons. Trying to overwrite them will throw an error rather than just doing well... nothing. Same goes for deleting certain things that aren't deletable. There are also more reserved words added that you cannot use and trying to redefine something like NaN, eval and arguments. Always a good idea to avoid using reserved and builtin objects as a variable or something like that.

While on the topic of eval, a variable declared in an eval statement will only be within that scope. It will not create a global variable. I personally never use functions like that and if I want a similar effect will use the closest thing to a macro in lisp, but that's all preference. You can also use the "use strict"; statement inside an eval, this making the code in it strict. Also, if "use strict"; has something like eval(...) in its scope, that code is strict. However if it's used like in a passed argument
function a(f, code) {
    "use strict";
    f(code);
}
a(eval, "var x"); // Not strict
a(eval, "'use strict';var x;"); // Strict

Which seems weird to me, but there's probably some reason for it I couldn't even guess. I'd have to guess it has something to do with scope, though.

Anywho, there's also stuff about arguments only containing the original arguments passed to a function. Makes sense. Also, arguments.callee is no longer used, so name your functions.

There was stuff on this, however I did not understand the wording, I'll edit this and explain it should I ever figure it out or should someone explain it to me better.

Now for the security it adds. function.caller and function.arguments is no longer global. This means you cannot access it globaly closing some security holes, I would hope obscure ones. Then there is the list of reserved words that are implements, interface, let, package, private, public, static, and yield. They did that because they will be for future versions or something like that. Quick side-note, ES5 unconditionally reserves the words class, enum, export, extends, import, and super, so avoid those as well as not all browsers may reserve them.

Now, do not rely on these "security" features in strict mode. Not all browsers support it, so those holes can still exist even if you use strict mode. Now problems I see arising even when it's widely supported is that some companies, one of which I've worked for, use old software for very long periods of time. My current job still has stuff using Windows 98 and I also worked for a place that still used IE5 when IE8 had been out for a while. Bottom line to me it seems, is never use Javascript for security, even for the user's security. It's running code that's arbitrarily downloaded.

Anyway, check out the link of MDN for a more detailed explanation made by people far smarter than I am on this subject if you want to know more, I just wanted to try and explain it myself as well.

Sunday, June 10, 2012

Refused to set unsafe header

*** If interested in CORS, try this.

This is an annoying little error that I've seen some people mention and even ran into with JavaScript. Moreover, it appears only in Webkit based browsers. So what's the problem? Basically a browser like Google Chrome will not change certain http headers in an XMLHttpRequest using setRequestHeader(). This is not a problem in Firefox or Opera from my testing so far. According to W3, it is actually the behavior they have as "standard" in their working draft. Headers they don't want you to overwrite are
"
  • Accept-Charset
  • Accept-Encoding
  • Access-Control-Request-Headers
  • Access-Control-Request-Method
  • Connection
  • Content-Length
  • Cookie
  • Cookie2
  • Date
  • DNT
  • Expect
  • Host
  • Keep-Alive
  • Origin
  • Referer
  • TE
  • Trailer
  • Transfer-Encoding
  • Upgrade
  • User-Agent
  • Via
Or if the start of header is a case-insensitive match for Proxy- or Sec- (including when header is just Proxy- or Sec-)."[1]
Now as to where this could be a problem, for me at least, is when I am trying to spoof some headers and want to do so in a browser as apposed to writing a script or getting an addon to do so for me. However, there is a security risk it could open up called HTTP Request Smuggling. As to how one would pull it off, I think an XSS vulnerability would need to be there as well for this in JavaScript to do anything. Bottom line for the security of it on the design end, make sure you check your sources before you use things like external scripts. As for user-end security, just be careful what sites you go to and use some common sense when you can.

Now as for ways around this, you would need either an addon to modify this for you or use related command line arguments. For example, to change the User-Agent header, you launch Chrome from your terminal with the option --user-agent="Some new UA string". As for an addon to do this, I do not know of any or if it is even possible, but if I find something I will update that here.

*[1] - http://www.w3.org/TR/XMLHttpRequest/#the-setrequestheader-method

Wednesday, May 9, 2012

WebGL Firefox Error

So I figured I'd post about this, since I ran into a problem and had to dig through some forum posts to find how to fix this. My problem was on Ubuntu 10.04 LTS using Firefox 12.0, with NVIDIA GeForce Go 6150. Trying to use WebGL gave me the error that I forgot to save because I'm smart like that.

Anyway, the quick fix was to enter in your url about:config, then search for WebGL, then enable webgl.force-enabled. No idea if it will cause any problems, but for now it works.

***Edit:
Updated to Ubuntu 12.04 LTS, same settings and everything and now WebGL just refuses to work at all. No solution found yet, all the ones that I tried did absolutely nothing.

Monday, May 7, 2012

Steam and non-Steam game on Linux

For a while now I've been messing around on and off with Wine and seeing what I can do with it, and I finally managed to not only install Steam, which I had done for a while now, but I installed the original fallout and registered it in Steam as a non-Steam game. So here's a quick walkthrough how, I did this on Ubuntu 10.04 LTS... so yeah.

First, you want to install Wine and PlayOnLinux. On Ubuntu you can just sudo apt-get install wine playonlinux.

Now you want to run PlayOnLinux, and select the "install" option. From there, search out Steam and install it. This should handle everything with Steam no problem.

After this is done, feel free to log into Steam, but close it out if you do for this next part. Steam is installed on a virtual drive, and therefore cannot see all your other lovely programs you may install with just Wine. So from here, you want to select "Configure" and go to the Steam virtual drive listed. From here you can alter Wine's settings for installation processes. For the original fallout, I had to set a virtual desktop and set Wine to Windows 95 due to the program running a version check and not letting it install in a newer Windows version. Options for all this are under the "Wine" tab, under "Configure Wine".

Now that all your settings are set for a successful installation, go to the Miscellaneous tab. There should be an options for "Run a .exe for this virtual drive". From there, run your .exe file to install. Once done, revert back any changes to the Wine configuration save for the necessary graphics changes you may have made. Now run Steam, select Steams add a game option, add a non-steam game and you should be able to find your game.

Should you wish to find the game to play directly with Wine, check in your home folder under .PlayOnLinux/wineprefix/Steam/Program Files/ for your stuff. If you want to have it in with your other Wine stuff, you can make a link and place it in your home folder under .wine/dosdevices/c:/Program Files/ (link the whole folder it's in, not just the .exe). This may be needed if it uses the drive it was installed on, due to the whole virtual drive stuff. You can then run it with a command like env WINEPREFIX="/path/to/home/.wine" wine C:\\windows\\command\\start.exe /Unix /path/to/home/.wine/dosdevices/c:/Program\ Files/path/to/game.exe. An alternative and easier method than messing around with that is to open up PlayOnLinux, then hit configure. From there, go into your Steam virtual drive and select the option "make a new shortcut from this virtual drive" in the General tab. From there, you can make shortcuts to launch your other games without going through steam, right from your PlayOnLinux window.

Hopefully that all works for other people, let me know should something break or have issues.

Sunday, April 29, 2012

2D Rectangular Collison Detection

I was just looking through some old code and came across something I couldn't quite work out in my head but had to find. Circles was very self explanatory, but rectangular shapes are a bit more awkward, so here's a simple way that will return either a true or false based on the idea that 0 is false and a number greater than 0 is true, assuming rectangle 1 is made of points a and b and rectangle 2 is made of points c and d.
Points relative to rectangle (showing collision)
We also assume each point is a list of the [x, y].

int checkCollision(int a[], int b[], int c[], int d[]) {
    int c1, c2, c3, c4, c5;
    c1 = a[0] < d[0];
    c2 = a[1] < d[1];
    c3 = b[0] > c[0];
    c4 = b[1] > c[1];
    c5 = c1 && c2 && c3 && c4;
    return c5;
}

That would (more or less) be the gist of detecting if 2 rectangles collide by finding if there is an overlap. The basic idea is make sure all the points are in an area they could  be in if there was an overlap. Then by making sure all are in an area they can be in if there is an overlap, detects the overlap, since all must be in those areas for an overlap to exist. For any other shapes that are not square, you must use trigonometry, which I may get into some day down the road.

Monday, April 16, 2012

MyBB nofollow external links

In the spirit of looking into SEO a bit and messing around with regex a lot more, I decided to do some SEO over on a forum I code at. They use MyBB. I dug up a plugin for this, however it was less than spectacular. The problem was it would add rel="nofollow" to internal links as well, and even replace anywhere in the message you saw target="_blank" with target="_blank" rel="nofollow". This can be a problem telling a crawler not to follow internal links, so I decided to re-write the plugin. It will not alter internal links and can recognize both http and https internal linking.

[Download v1.0]

Maintained: https://github.com/anarchywolf46/MyBB-Plugins

Thursday, April 12, 2012

Minecraft /dev/input/event error

I've been playing Minecraft for a while, and having a computer with Ubuntu, that's what I play it on. However I kept running into a series of error messages about /dev/input/event. Since everything ran fine, I never bothered to see what the error was about. Today I finally searched around and it was a problem with file permissions. A quick command to fix it is sudo chmod go=u /dev/input/event*.

A quick explanation of what it does if you don't understand it, it will copy the permissions for the root (u)ser to the (g)roup and (o)ther, other being anyone else not in the group or that user.

Tuesday, April 3, 2012

CSS3 ::selection

While working on some stuff I cam across a CSS3 pseudo-class selector called ::selection. While this is something you can do in javascript, it makes things a lot easier when styling can all be done in CSS rather than javascript workarounds. The javascript equivalent would be rather annoying to do. So the CSS3 way, as with any CSS, is rather simple and quick to do, minus a few catches here and there. CSS3 still lacks a good deal of support and isn't good if you want some real backwards compatibility, however I'm more interested in how to use this to just screw with things in ways it could be exploited, bad or just annoying and this particular property hit right on the head for some annoying fun.

You may wonder how this could be annoying, and quite simply, think about what would happen if selected text disappears. It would no longer be selected and then have to reappear, unselected. Thus the end result is text you cannot highlight without altering the CSS. So the code for that looks something like this.
::selection {
  display: none;
}
::-moz-selection {
  display: none;
}

I've only tried this in Firefox 10, but when I do it on Firefox, the text is impossible to highlight, however I can drag the text. As to the effects it will have in other browsers that support it, well that's all part in the fun in trying to mess with things to get results like this.

Try to highlight this and see what happens if your browser supports it.

Tag Cloud

.NET (1) A+ (2) addon (6) Android (3) anonymous functions (5) application (10) arduino (1) artificial intelligence (2) bash (4) c (7) camera (1) certifications (4) cobol (1) comptia (4) computing (2) css (2) customize (16) encryption (2) error (19) exploit (17) ftp (3) funny (2) gadget (2) games (2) Gtk (1) GUI (5) hardware (6) haskell (15) help (8) HTML (6) irc (2) java (5) javascript (21) Linux (19) Mac (4) malware (2) math (8) network (9) objects (2) OCaml (1) perl (4) php (9) plugin (7) programming (42) python (24) radio (1) regex (3) security (25) sound (1) speakers (1) ssh (3) story (1) Techs from the Crypt (2) telnet (2) tools (14) troubleshooting (5) Ubuntu (4) Unix (4) virtualization (1) web design (14) Windows (7) wx (2)