Influencing the topology

[Kai Lingemann]

Hello,

recently, I started playing around a bit with some neural network
libraries, ran across FANN, and really came to like it!

However, I'm missing a way to influence the net's topology - either
by inserting/deleting connections/neurons, or by specifying a certain
topology by hand during creation of the network.

There are some mails about this matter in the mailing list's archive,
but these are already 2 years old. I find it hard to believe that nobody
spent any effort in implementing such functionality (even if it somehow
didn't make it into the official branch), because it would open FANN
for the whole world of automatically generating an optimal network,
"optimal brain damage", and the like.

Any newer input about this question?

Thanks a lot in advance,

Kai


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

[Chris Spencer]

You can specify the number and size of layers during network creation
by using fann_create_standard_array
(http://leenissen.dk/fann/html/files/fann-h.html#fann_create_standard_array).

There's also limited support for the Cascade Training algorithm, which
dynamically builds the network while training
(http://leenissen.dk/fann/html/files/fann_cascade-h.html). I too would
like to see more emphasis placed on learning optimal network weights
*and* topology. However, understand that dynamic network topology is
still an area of intense research, and so there's still no proven
"optimal" answer.

Regards,
Chris

Post by Kai Lingemann
Hello,
recently, I started playing around a bit with some neural network
libraries, ran across FANN, and really came to like it!
However, I'm missing a way to influence the net's topology - either
by inserting/deleting connections/neurons, or by specifying a certain
topology by hand during creation of the network.
There are some mails about this matter in the mailing list's archive,
but these are already 2 years old. I find it hard to believe that nobody
spent any effort in implementing such functionality (even if it somehow
didn't make it into the official branch), because it would open FANN
for the whole world of automatically generating an optimal network,
"optimal brain damage", and the like.
Any newer input about this question?
Thanks a lot in advance,
Kai
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Fann-general mailing list
https://lists.sourceforge.net/lists/listinfo/fann-general

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

[Steffen Nissen]

Hi Chris,

Just curious, why do you say "limited" support? What is missing?

Steffen

Post by Chris Spencer
You can specify the number and size of layers during network creation
by using fann_create_standard_array
(
http://leenissen.dk/fann/html/files/fann-h.html#fann_create_standard_array
).
There's also limited support for the Cascade Training algorithm, which
dynamically builds the network while training
(http://leenissen.dk/fann/html/files/fann_cascade-h.html). I too would
like to see more emphasis placed on learning optimal network weights
*and* topology. However, understand that dynamic network topology is
still an area of intense research, and so there's still no proven
"optimal" answer.
Regards,
Chris

Post by Kai Lingemann
Hello,
recently, I started playing around a bit with some neural network
libraries, ran across FANN, and really came to like it!
However, I'm missing a way to influence the net's topology - either
by inserting/deleting connections/neurons, or by specifying a certain
topology by hand during creation of the network.
There are some mails about this matter in the mailing list's archive,
but these are already 2 years old. I find it hard to believe that nobody
spent any effort in implementing such functionality (even if it somehow
didn't make it into the official branch), because it would open FANN
for the whole world of automatically generating an optimal network,
"optimal brain damage", and the like.
Any newer input about this question?
Thanks a lot in advance,
Kai

-------------------------------------------------------------------------

Post by Kai Lingemann
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share

your

Post by Kai Lingemann
opinions on IT & business topics through brief surveys-and earn cash

http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Post by Kai Lingemann
_______________________________________________
Fann-general mailing list
https://lists.sourceforge.net/lists/listinfo/fann-general

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Fann-general mailing list
https://lists.sourceforge.net/lists/listinfo/fann-general

[Chris Spencer]

Sorry, I meant to say there's limited support for dynamic topologies
*with* Fann's implementation of the Cascade-Correlation architecture.
I don't believe there's anything missing. However, as far as I know,
aside from initial network construction and cascading training, the
user has little direct control over network topology through the API.

Regards,
Chris

Post by Steffen Nissen
Hi Chris,
Just curious, why do you say "limited" support? What is missing?
Steffen

Post by Chris Spencer
You can specify the number and size of layers during network creation
by using fann_create_standard_array

(http://leenissen.dk/fann/html/files/fann-h.html#fann_create_standard_array).

Post by Chris Spencer
There's also limited support for the Cascade Training algorithm, which
dynamically builds the network while training

(http://leenissen.dk/fann/html/files/fann_cascade-h.html).
I too would

Post by Chris Spencer
like to see more emphasis placed on learning optimal network weights
*and* topology. However, understand that dynamic network topology is
still an area of intense research, and so there's still no proven
"optimal" answer.
Regards,
Chris

Post by Kai Lingemann
Hello,
recently, I started playing around a bit with some neural network
libraries, ran across FANN, and really came to like it!
However, I'm missing a way to influence the net's topology - either
by inserting/deleting connections/neurons, or by specifying a certain
topology by hand during creation of the network.
There are some mails about this matter in the mailing list's archive,
but these are already 2 years old. I find it hard to believe that nobody
spent any effort in implementing such functionality (even if it somehow
didn't make it into the official branch), because it would open FANN
for the whole world of automatically generating an optimal network,
"optimal brain damage", and the like.
Any newer input about this question?
Thanks a lot in advance,
Kai

-------------------------------------------------------------------------

Post by Chris Spencer

Post by Kai Lingemann
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share

your

Post by Chris Spencer

Post by Kai Lingemann
opinions on IT & business topics through brief surveys-and earn cash

http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Post by Chris Spencer

Post by Kai Lingemann
_______________________________________________
Fann-general mailing list

https://lists.sourceforge.net/lists/listinfo/fann-general
-------------------------------------------------------------------------

Post by Chris Spencer
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share

your

Post by Chris Spencer
opinions on IT & business topics through brief surveys-and earn cash

http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Post by Chris Spencer
_______________________________________________
Fann-general mailing list
https://lists.sourceforge.net/lists/listinfo/fann-general

--
Steffen Nissen - http://MySpace.com/SteffenNissen
Project Administrator - Fast Artificial Neural Network Library (fann)
http://fann.sf.net
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Fann-general mailing list
https://lists.sourceforge.net/lists/listinfo/fann-general

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

[Kai Lingemann]

Hi Chris,

thanks for your answer.
Well, I read the documentation about FANN-creation beforehand, including
the very nice option of cascade training.
I do understand the difficulty (impossibility?) of finding an optimal
network, and naturally, I don't expect to find any implemented algorithm
for this problem. However, it surely would be nice to have a way to
specify or change the topology in more detail (which can't be that
hard), in order to make it possible for the user to implement some
ways of "improving" the topology on his/her own.

Don't get me wrong - I do know that the library is non-commercial, and
nevertheless a wonderful, intuitive tool. No criticism on my part here.
I just wanted to check whether any reader of this mailing list has any
knowledge of/implementation with improved funcionality, beyond the
downloadable code, that would enable such improved user-definable
topology.


Regards,
Kai


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

[Steffen Nissen]

Hi Kai,

I have had plans to implement this into library, but unfortunately I have
never gotten around to it. There do however exist some functionality to do
so within fann_cascade.c, which might help anyone wanting to implement this.

Please take a look at the fann_add_layer and the fann_add_candidate_neuron
functions in fann_cascade.c.

Unfortunately the structure of the ANN library is designed for speed and not
for flexibility, so changing the topology of the network can be a bit
tricky, but in no way impossible.

Steffen

Post by Steffen Nissen
Hi Chris,
thanks for your answer.
Well, I read the documentation about FANN-creation beforehand, including
the very nice option of cascade training.
I do understand the difficulty (impossibility?) of finding an optimal
network, and naturally, I don't expect to find any implemented algorithm
for this problem. However, it surely would be nice to have a way to
specify or change the topology in more detail (which can't be that
hard), in order to make it possible for the user to implement some
ways of "improving" the topology on his/her own.
Don't get me wrong - I do know that the library is non-commercial, and
nevertheless a wonderful, intuitive tool. No criticism on my part here.
I just wanted to check whether any reader of this mailing list has any
knowledge of/implementation with improved funcionality, beyond the
downloadable code, that would enable such improved user-definable
topology.
Regards,
Kai
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Fann-general mailing list
https://lists.sourceforge.net/lists/listinfo/fann-general

[Chris Spencer]

Another approach to dynamic topology you might find interesting is an
evolutionary method called Neuro-Evolution of Augmenting Topologies,
developed by Ken Stanley at the University of Texas. He's released his
C++ code under the GPL and there are a few other re-implementations in
C# and Java.

Regards,
Chris

Post by Steffen Nissen
Hi Chris,
thanks for your answer.
Well, I read the documentation about FANN-creation beforehand, including
the very nice option of cascade training.
I do understand the difficulty (impossibility?) of finding an optimal
network, and naturally, I don't expect to find any implemented algorithm
for this problem. However, it surely would be nice to have a way to
specify or change the topology in more detail (which can't be that
hard), in order to make it possible for the user to implement some
ways of "improving" the topology on his/her own.
Don't get me wrong - I do know that the library is non-commercial, and
nevertheless a wonderful, intuitive tool. No criticism on my part here.
I just wanted to check whether any reader of this mailing list has any
knowledge of/implementation with improved funcionality, beyond the
downloadable code, that would enable such improved user-definable
topology.
Regards,
Kai
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Fann-general mailing list
https://lists.sourceforge.net/lists/listinfo/fann-general

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

[Kai Lingemann]

Chris, Steffen,

thanks a lot for your hints/ideas!

I'll take some closer look into them tomorrow (it's getting a bit late
today :-) ) ... just wanted to say thanks.

Regards,
Kai

PS: I'm amazed with the speed that you get (useful) information in this
mailing list!

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

[Steffen Nissen]

Apparently there are a lot of guys on this mailing list, who sit in-front of
their computer on a Friday night ;-)

Post by Kai Lingemann
Chris, Steffen,
thanks a lot for your hints/ideas!
I'll take some closer look into them tomorrow (it's getting a bit late
today :-) ) ... just wanted to say thanks.
Regards,
Kai
PS: I'm amazed with the speed that you get (useful) information in this
mailing list!

-------------------------------------------------------------------------

Post by Kai Lingemann
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Fann-general mailing list
https://lists.sourceforge.net/lists/listinfo/fann-general

[Chris Spencer]

For the record, I'd like to say I'm typing with *both* hands ;)

Chris

Post by Steffen Nissen
Apparently there are a lot of guys on this mailing list, who sit in-front of
their computer on a Friday night ;-)

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

[Josh Menke]

Hey, anyone want to answer this quick without me having to look it up
(feeling lazy on a Friday night at the airport).

If I want to have all of my inputs go not only to the hidden layer (1 hidden
layer) but also my output layer in order to catch the "linear" relationships
faster, how would I do that in fann?

Thx,

--Josh

Post by Chris Spencer
For the record, I'd like to say I'm typing with *both* hands ;)
Chris

Post by Steffen Nissen
Apparently there are a lot of guys on this mailing list, who sit

in-front of

Post by Steffen Nissen
their computer on a Friday night ;-)

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Fann-general mailing list
https://lists.sourceforge.net/lists/listinfo/fann-general

[Steffen Nissen]

yep,

http://leenissen.dk/fann/html/files/fann-h.html#fann_create_shortcut

Post by Josh Menke
Hey, anyone want to answer this quick without me having to look it up
(feeling lazy on a Friday night at the airport).
If I want to have all of my inputs go not only to the hidden layer (1
hidden layer) but also my output layer in order to catch the "linear"
relationships faster, how would I do that in fann?
Thx,
--Josh

Post by Chris Spencer
For the record, I'd like to say I'm typing with *both* hands ;)
Chris

Post by Steffen Nissen
Apparently there are a lot of guys on this mailing list, who sit

in-front of

Post by Steffen Nissen
their computer on a Friday night ;-)

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Fann-general mailing list
https://lists.sourceforge.net/lists/listinfo/fann-general

--
Joshua Menke
Machine Learning Scientist
Trust and Safety Applied Research
ebay, Inc
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Fann-general mailing list
https://lists.sourceforge.net/lists/listinfo/fann-general