Hi everyone - need a bit of a consult on this. I went back to my books at the weekend to check up on this, and still didn't get a clear answer, so would be interested in your thoughts.
When introducing a VSR, I always thought (and was told by Emma) that you still click for the behaviour, but don't necessarily reward each click. Having consulted the books (AK's 'CT for your horse', S&V Karrech 'You can train your horse...', KP's 'DSTD' and Burch's 'How Dogs Learn') they seem to be saying that you don't click each behaviour, just click randomly. I am confused - help please!
Hee Hee. Glad you brought this up Sam.I was trying to think through the very same thing a few days ago.
I know when Marie was working with Crunchie and she had gone on to a new bhvr his offering his leg got more intense as if to say 'look mum I'm doing it'
She wasnt clicking each of these trys.However she wasnt giving the horse any clues to say that that ' you trying harder' was right.Maybe if you wernt to click at all depending on how your horse thinks he would just move onto a different bvr and try that for clicks.So perhaps clicking for each try but no reward at least lets him know he is on the right bhvr.
EEEEk!
Millie, Catherine, Marie you there????
This is something that is playing on my mind too. As you know I click for the correct bhr but don't always give a reward/treat (on VSR of course). It seems to work well with Crunchie. But I keep reading different interpretions on what should be done for VSR. I'm sure Karon Pryor talks about treatless clicks in DSTD, but when I read 'How Dogs learn' they had a different method of VSR.
I'm finding it ever confusing with the ridden work (walk)- as you know Crunchie was stopping every time I clicked, to wait for a treat, when he didn't get one he sets of again - but then I realised that I am using two types of VSR - no. of steps offered and intervals of clickless treats, so I have gone back to treating each click and using 'Good Boy' and a scratch to encourage him in the right direction when I'm doing the ridden work. This seems to be working well, but then I am still using treatless clicks on other bhrs we are working on - he seems to understand what I want though.
Remember, Ben said that VSR can get really complicated - see what he means now!
Have you got something in mind that you want to apply VSR too? I 'm not sure what Mali would be like as Crunchie is such a contrast to her, but I do think whether I offer treatless clicks or not would depend on the bhr I was training - obviously treatless clicks are not working very well with the ridden work with Crunch, but work on the ground seems to progress well with this method...
You're not allowed to ask me like that as then people will expect me to talk some sense when I respond!
But I can try...
Surely in the first stages of learning the first behaviour put on a VSR it would be really confusing to the animal to have random clicks. When everything it's learnt previously would have been click/treat to mark something really specific? So wouldn't it make more sense to give a bit more help?
OK, so I'll go out on a limb and use Jak as an example for you all to help me with! As I said to some of you privately we were working on head down (un-cued) and he cottoned on to it really quickly. We found it a fairly easy thing to free-shape, although with the occasional moment of frustration when he would stretch one of his front legs in an exaggerated and slow-motion pawing action. This is typical for him but I sometimes accidentally click just as he starts to do it (grrr!!) and so then have to go through a cycle of only clicking head-down when all 4 feet are on the ground.
But back to the VSR(!) - I started trying to put this on a variable schedule, with my initial plan being to get two head-downs in sequence. He'd do the first head-down and then get cross when I didn't reward him (although I did click). In retrospect it seems very obvious to me that the reason it wasn't very successful was that I wasn't relaxing the criteria enough - I was expecting two full head-downs and because I wasn't getting the second one I didn't click. Looking back (and we will start trying this again now) I am sure there would have been a tiny nod I could have jackpotted to help him get the idea of doing it a second time.
So with this being our first ever attempt at a CT variable schedule I am sure that had I been trying to click randomly I would have confused him even more than I did - I'd have ended up shaping all his methods of showing his frustration (and I don't need any help with that!!) But if I build up by shaping the 2 nods, then 3 and then gradually build up to any number then maybe one day he would be able to tolerate it. But not yet - unless I've misunderstood what you meant by random clicking.
Looking at it from another aspect though - maybe all the sources you quoted, Sam, are saying the same thing really. So say I wanted 2 head-downs - I could either click when the head is at the lower-most point on each repetition or I could shape the whole movement so that he gets clicked for going down, coming back up, going down again etc So in that case it could appear to be random clicking but in actual fact you are clicking tiny little parts of the overall behaviour.
Think this post is my most rambling so far - so hope it vaguely makes sense!
Has anyone else got some examples that could help explain this one? On the understanding that we learn from each others' successes/mistakes but don't try to replicate the situation (which obviously probably wouldn't work)
OK, so I'm probably the worst person in the world to advise on this as Arnie is Mr Toys-out-of-the-pram if things don't go his way (ie if he doesn't get a treat every time he thinks (note HE thinks) he has earned it, BUT.....
I asked Ben about this ages ago as I was really confused about when to introduce variable schedules. He said you have to get the first stage of behaviour being offered, then you put it onto a variable schedule (ie clicking each time but not treating), then you can raise the criteria by only clicking the bigger tries, then variable again etc. When you have the final behaviour as you want it, then you can introduce a cue. I hope I've got this right. I haven't done much variable stuff with Arnie yet, though Ben said it would be very important to help with his biting (slap on the hand for me). Sometimes he seems to get it ("oh, you want me to do it again...?") and other times he just gets very cross ("oi, WOMAN!! Can't you see me standing on my nose with my leg round my ear?? Treat! Now! Or else..."). I know I need to work on it more...
Oh and apologies if this hasn't answered the question properly, I'm at work and just skimming through these!
I think Catherine makes a good point about the clicks appearing to be random when really they are well timed, but variable.
If I understand this VSR thing correctly (and there's always the chance I don't!), I think, and as Catherine said, we have to be careful about asking for a higher criteria than we have trained prior to the VSR (ie, Jak and his head nods).
If I were training a horse to lift its foot to a certain height, once the horse understood lifting its foot to the required height was getting the click/treat, I might go something like:
Millie, that's sort of what I do with Crunchie. Sometimes I push it too far - as some of you saw when I had my home visit with Ben, I pushed crunchie's touching to 7 treatless/clicks but I went to far - although I can get to six no probs. I tend to just feel how far to go sometimes - the cone touching is the furthest I've gone with VSR though.
Becasue Crunchie is very bold, if I gave him a treat each time I clicked (forever) I think I would create a very 'in your face' and probably biting horse - I suppose I've managed to get to the point where he listens for the click on the off chance he might get a treat, if he doesn't get a treat he does the bhr again but usually more enthusiastic (sometimes too enthusiastic!).
When I fist introduced the treatless click, Crunchie did get very pushy with me as if to say 'Oi where are the treats woman' but then as soon as he showed signs of doing the bhr again (even if he didn't complete it fully) I clicked and treated - it didn't take long for him to realise that if a treat didn't follow a click he had to try again.
Every horse is different though, and I can imagine some horses loosing interest when using treatless clicks.
I suppose you just have to make sure that you introduce the variable schedule gradually, ie. not missing out too many click/treat pairings in the beginning. Just start with one missed treat, until they are used to that concept and they don't panic about the treat not following the click as they're used to, then build up to two, three etc.
It's important though, I think, to vary it and not just keep building and building, or as Marie says they might lose interest. It's the gambling theory again isn't it, you never know when you might win... So to follow on from what Millie was saying, you could go from missing one treat, to missing two, then back to click/treat, then missing one, then three, then two, then none etc. etc. The ideal is, as Marie says, to have them listening for the click and hoping there might be a treat but not expecting it as their god-given right...something Arnie and I haven't quite "clicked with" yet...LOL!
I hope I am the correct lines here, sorry if I am not!!
I understood that when you click you must reward. If I click on something by mistake, I always reward it. The click/treat association is in a way a conversation in which you tell the animal 'when you do what I would like, you will get what you want and that magic clicker sound tells you that the treat is coming'. This to me is the training bargain. For the bargain to keep its force, both of you have to live up to your part of it. If you do not always treat after a click the animal could get fly to this and perhaps lose interest. Also, we are working on +reinforcement, isn't not treating after a click -ve reinforcement? Instead of not treating after a click, how about clicking then rewarding after maybe a count of 5. That way you getting the animal to think what's going on but you then honour your bargain.
If I was working on a variable schedule, I would be clicking after eg 1 good step then 5 good steps, 2 good steps, 7 good steps. After many sessions I would up the criteria to say 5 good steps, 1 step, 9 steps,
7 steps, 13 steps and so on. The animal is unaware of when you are going to treat so keeps his performanc high in the hope of it coming
Last posting, -ve reinforcement should read -neg punishment. I edited several bits of the message I sent but it didn't seem to work. I hope you understand it!
That's a really interesting point Fiona, and one that had never occurred to me. I always used to think that there was no way of applying Neg Punishment in animal training (can't say to a horse "You didn't jump the big fence so I'm not going to turn you out today") but if my understanding of Neg P is correct - the witholding of something pleasant - (please correct me if I've got this wrong) then as you say, not treating after the click is Neg P. I'm going to have to think about that one some more!
But back to VSR. My initial understanding of it was the same as yours Fiona, and that seems to make more sense. However (just thought of this) what about the clicker as conditioned reinforcer? I have said to some of you before that when I was using the clicker while I was riding, after a while Mali would stop bothering to turn round for her treat, as if the click itself was reward enough. Does it become reinforcing for the animal to be told "Yes, right answer"?
I suppose if you click each correct bhr, but don't treat each click, at least you're telling the horse that it's done the right thing - the variable bit comes from varying whether or not you're treating, so the horse keeps offering the bhr because it knows that's the right thing, but is 'gambling' on whether this is the time that will earn the treat. Does that sound about right?
The way I have attempted to use VSR is to with hold the click, I always reward when I click, for example when I was teaching backing up, I initally rewarded just a shift in weight then 1 step, and now I can get Apache to back about 6 or 7 steps with the click being given when I want him to stop the behaviour.
Just a thought - does this tie in at all to whether the click ends the bhr? When reading DSTD, KP seems to be saying that the click ends the bhr, but this obviously isn't true in all cases. If you're training something that requires the horse/animal to be on the move, eg lungeing or riding, then you don't want it to stop when you click - perhaps this is why some people don't click at each correct bhr, but sort of spread the clicks out.
Actually, I'm not all that sure all that made sense, but perhaps you know what I mean!
I've actually used the same kind of variable schedule as Marie, in that I click for each correct bhr, but don't always necessarily treat for each one. Esme seems to be okay with this.
Hi Jules. Isn't that more raising the criteria than introducing VSR? When you raise the criteria, you withhold the click to encourage the horse to try harder/something else, but with VSR you want the behaviour repeated (and it will be clicked each time to show that's right), but the horse won't know when it will get the treat so it is motivated to keep offering the behaviour. Then eventually you can make the gap between treats long enough that you don't have to have a pocket bulging with treats forever (!) and can then also phase out the click and replace it with a stroke/scratch/word of praise or whatever's convenient with the occasional treat to keep up the behaviour.
Hi Pam. Having the horse stop when you click is something I've been thinking about recently. I'm not riding Arnie yet, but as he has a habit of stopping dead and planting in the middle of the road when we are out on walks, I've started getting out my clicker and clicking him when he is walking well to try to get the message across! I like to stop him to give him a treat as he doesn't like getting treats "on the move" (!!) and can get a bit snatchy. But I didn't want to be rewarding him for stopping! So on advice from Ben, I click and then ask him to whoa, then I treat. Most of the time it works fine and he happily walks on again afterwards. But I have to be careful when doing it just after a planting session (ie when I've FINALLY got him to walk on and want to reward him for walking) or the action of stopping begins another session of standing there like a concrete donkey...
So perhaps introduce the idea of only treating after you have asked the horse to "whoa", to solve the screeching to a halt after the click problem?? But then again, would this be too much of a gap between the click and the treat? Hmm....
We were talking to Kay Laurence, a top clicker trainer for dogs, recently, and she mentioned that agility dogs have a "marker" they go to after they have done a behaviour to get the treat, so they keep moving and don't immediately stop dead. Perhaps this could be adapted for horses too, I don't know. I would think that introducing VSR would mean that the horse SHOULD carry on moving as it wouldn't know whether it would get a treat or not.
Must admit I never click without treating, but what i also do when I have asked for a behaviour (an established one)is tell him "good boy" and give him a scratch rather than a click and treat.
Hi millie,
I would fade out the click once the behaviour was sound therefore you wouldn't need to treat. Click to me = reward. If the behaviour ever diminished I would go back to C/R to refresh it.
Hi Pam
I read somewhere about separating the click and the reward point. If you delay the reward as you say in a moving action it gives you the oportunity to keep a behaviour going, the horse gets a double oportunity for reward, click to say yes you are right and then on the food delivery. And it strengthens the behavior. It also said that the animal needs to get used to this gap of reward as most of CT is done with click ends behaviour
Claire, Ben said the same to me which is OK if your horse does keep going after the click, unfortunately Crunchie won't. I don't see it as rewarding the halt, as if the horse has been trained to know that the bhr at the click is the right bhr then the halt comes after that point if you see what I mean. Also, yes I am using a 'marker' with Crunchie, the 'Goodboy' and scratch encourage him to continue what he is doing - seems to be working well at the moment. As said previously, he will work for treatless clicks without a problem when we work on the ground, but Ben pointed out that when I do the end click I probably go towards my bag for a treat, letting him know a treat is coming.
I see what you mean about the punishment thing, but if the click is the reward, surely the treats are something extra? I work on the basis that the click is what my horse is working for not the treat (if he gets a treat it is an added bonus, and the possibility of getting a treat keeps him motivated to continue what he is doing).
I tend to think that if you treat everytime you click you will devalue the the treat - ie if every time you went on the lottery, you won £10 million, the thrill would eventually wear off. The fact that people don't know whether it 'could be them' keeps them going and eventually they can become addicted, to me this is what keeps the horse motivated to work for the click, just in case they get a treat.
Also, would you completely fade out the click and treat? Or would you click and treat every now and then to keep the bhr going?
Would have liked to post before now but have busy with work!
On the treatless clicking - this seems to be something that some trainers say not to do but what a few of us have been taught. If you don't use treatless clicks then am I right in thinking you need an additional "keep going" signal? We were talking about this with Ben on the CT workshop and he said that as horses are so good at picking up on body language they could distinguish between a click as a keep going signal and a "click + hand going into bum bag for a treat" as a reward. So as I understand it that's why it is possible to use treatless clicks and so they won't be seen as -P.
Fiona - yes, you can only do this if you have a high enough rate of reinforcement that the animal doesn't lose interest. The way you talk about building up a VSR is exactly how I started off trying with Jak (my horse) and it also seems to be very similar to the Alexandra Kurland Pigeon Peck exercise (see http://www.theclickercenter.com/news/news03b.html). What I found with this is that if he did a head-down that wasn't clicked or rewarded then he lost interest. This morning I tried it more as Millie suggested above - C/T x about 10, then click+no treat, then lots more C/T, then click+no treat etc. So he barely noticed that there were a few treatless clicks thrown in and he remained interested, especially as I slipped in a few mini-jackpots before and after the treatless clicks. So this way definitely worked better for us.
Millie - as I just said above - your suggestion worked so much better than how I'd been trying before so we're off and away now. It was really interesting that once I'd remembered to relax the criteria he started not lowering his head quite as much but he was doing lots of them and consistently. Then I was a split second late with the click and you could see him think about it and lower his head right to the ground. Then after that he was doing the really nice low ones consistently. What a clever pony! I've had to settle for a little bit of simultaneous foot tapping but am phasing out the full-blown pawing so I'm hoping he'll finally twig that the behaviour doesn't involve a leg flapping around!
In your example above - I take it you wouldn't try two consecutive treatless clicks within the first session but build up to that gradually?? I think that's probably what I should do but would be interested to hear if you've managed otherwise.
Sam - I agree with what you said about the click becoming a conditioned reinforcer but I'm not convinced about Mali working for the click alone unless it was a REALLY well-established behaviour which had been on a VSR. Could she have just slipped into a -R mentality?
Millie - I'm not sure I agree with you about not being able to phase out the treat if you don't use treatless clicks as this is what all the AK people seem to do (so I understand it anyway). They build up their VSRs with the pigeon peck exercise and phase it out that way. I'm not saying I agree with that and it already seems as though it's not suited to Jak/me but it seems to work for them. But I think it would be harder to set up behaviour chains that way.
Pam - I think the click does end the behaviour but when you use it as a keep going signal you are only ending intermediate bits of the behaviour, not just clicking randomly.
Claire/Jules - think this is still a VSR the way AK teaches it but the way Ben seems have taught us, yes, it is how you would improve the behaviour. So the way I understand it you could use treatless clicks to "keep going" the backwards steps or withold the click to get bigger or faster or whatever steps. Interesting what you say about the "marker" Claire, will have to think about that for longer! I think I agree that if you have a VSR then the horse should keep going and not stop until it sees (or feels, if you are riding) your hand move towards the treat bag.
Fiona - Please could you elaborate a bit here - if you have a sound behaviour would you fade out the treat first and the click later or fade out the click and treat simultaneously? If you consider the click to be a reward at this stage then isn't it the same as a treatless click as we discussed above?
Phew! Well that went on a bit. Sam, do you still prefer this format? It's times like this when being able to reply to each individual post might be easier?!
CatherineB
I'll try!!
Because I don't separate the click and reward, I would fade both out simultaneously. I don't consider the click the ultimate reward (although it must be rewarding to a certain degree because the horse then will be thinking about his end reward) I consider it as a marker to say 'well done you were spot on / or moving in the right direction....your reward will follow'.
Once I've untied the knots in my body and brain I am going to ask a really easy question!
Hi Catherine, I do sometimes go for two consecutive treatless clicks in the first session of VSR and build from there in a suck it and see kind of way (not very scientific or helpful, having a bad brain day today!).
I have read this post with much interest! I have a question, how do you fit in Fixed Ratio Schedule?
The way I have overcome the VRS problem that has been mentioned is.....
Train a new behaviour on a Continuous Schedule (click and treat each desired behaviour) then once established, ask for 2 repetions before click and treat. Once FRS2 is being fluently offered I then more to 3 etc upto to approx 5 then go to a VRS the horse then undersands the concept of 'I may have to do this once or five times before I get Click + Treat'. I have found you don't need to stay on a fixed ratio for very long and it prepares the horse for a variable. This is the point when the motivation and learning gets mind blowing.
To move to the next sucessive approximation step I just withhold the Click and wait for something new, if it doesn't happen the step is too big and I look at how I can break it down further.
Not sure if this helps??? or am I just talking naff!!!
That was really interesting. What you suggested was much more along the lines of what I had been trying. We had head down easily and as far as I was concerned it was time to move up to 2-in-a-row. But Jak got really frustrated with this as he wasn't getting rewarded enough to keep his enthusiasm. Do you think this depends on the horse or has mine just wrapped me around his little hoof?? In the last couple of sessions I've found that we can get 2 or 3 if I slip them in as Millie suggested. I guess for us now the next stage is to phase out the individual ones so that we are consistantly doing 2. Then start doing the occasional 3 slipped in, then phase out the 2s. So a combination of what you and Millie say seems to be how it is working for us at the moment
My next question is.....
As I said, we have been gradually stretching our VSR and he will now tolerate quite a few unrewarded repetitions of head-down. But even when he is getting it right he has a tendancy to paw (or even just tap) his foot. This is intermittant whereas the head-down is consistant so I think he knows what behaviour I'm asking for, but I am also worried that I am reinforcing the leg action as well. At some point I need to start only rewarding him when all 4 feet are on the ground but if I with-hold the click then he paws all the more vigorously (simultaneously doing a beautiful head down so I end up clicking to cheer him up! smack wrist!). So should I continue to stretch the VSR and accept that the pawing is a means of showing frustration which should extinguish when he understands VSR a little better. Or should I start working on "4 feet on the ground"?
Catherine, my gut feeling on this is to go back to a continuous schedule (is that what it's called?) and eliminate the pawing - ie only click when all 4 feet are on the ground. As you say the pawing could be a sign of frustration, or it could be that Jak still isn't completely clear - if he gets clicked when he paws and clicked when he doesn't, he will never get to learn the difference. I would click just after one head down, but only when all four feet are still. Can't quite explain why I think this, but that's what I'd do.
I agree with Sam's gut reaction. Can you explain in further detail how you taught head down? Was it with targets, pressure /release? I agree a smacked wrist is in order! Jak may be getting confused with your required behaviour and thinks that the pawing is needed when you withold the click and the pawing becomes more intense I feel this is an extiction burst (a last ditch attempt at getting the reward) out of fustration. However, I may be very wrong! I would, as Sam suugests, go back a few steps and run through the training again to consolidate the learning process ensuring that 4 feet still is the requirement. What is your aim for the head lowering exercise? If it is a calm down cue then foot up also may benefit (not pawing but as if you were going to pick his foot up to pick it out). I would also give the headlowering a break for a few sessions and move sideways to a different task like touch - push a football or something Jak would find 'fun'.
I have read the posts again and just wanted to add a couple of things. The Clicker to me is a Conditined Reinforcer, a bridging signal that means click = treat on its way for the good thing you just did. The clicker has to be 'charged up' before the association of behaviour = click = treat is made. If treatless clicks are given in the early stages of CT then the clicker starts must become re-associated with behaviour = click = nothing, which is as suggested in earlier posts -R. And the behaviour become extinguished in the absolute worst case?
Through repitition behavior = click = treat becomes so powerfully associated and as only small amounts of reward are fed on the whole, the treat naturally becomes just the 'thought' of eating and the horse starts to not worry about getting his reward. Hence why some stop looking for the treat and just go for the behaviour again. It does take some time for association to be build that click = +R alone. horses, having different learning styles themselves, will take more or less time get to this stage.
Fade outs, I have been using a combination of different things with different horses. Some i have used CT to build and association of behaviour = (2ndry +R) scratch and others a distinct word such as 'good'. For example behaviour 'good' click = treat. I then fade out the clicker then the treat and substitute with a scratch. I did this very fast with my mare, the word 'good' = scratch now means as much to her as click = treat. But as good and scratch are both secondary reinforcers it is important the they are associated with a primary of say food before they will have the same effect. I have found that now if something new comes along and I have no clicker or treats I can get the same results from using the good = scratch method.
So to conclude! I don't feel that treatless clicks are always effective at fading out click = treats but by substituting them with a word (rather than a cue) and adding 2dry +R it will keep the motivation better in some horses. What do you guys think??
Thanks Sam, Dee - ok will give it a try. He's learnt head-down purely with free shaping, no pressure or targets. My aim is to just give him an exercise to do so that we can extend our repertoire of behaviours (and long-term I would incorporate it into ridden work to stretch through the back). I agree with what you're saying and would say the same if someone asked me the same question. But a little voice inside is saying "but in this case it's different" - isn't that just the most annoying thing for someone to say to you?! So I think I will do as you suggest but train something else for a while as we've done quite a lot of head-down work recently and I think if I re-start it there is a danger of him getting bored (which could also be the reason for the leg stuff in the first place). When we go back to head-down I'll be strict about it!
Interestingly I started trying to start something new over the weekend. I was planning to start shaping a circle and was really just trying to see what would happen so that I can start making a proper shaping plan based on what he would give me. In the end it wasn't too successful and step 1 of the shaping plan needs to be "stand and do nothing" which was what we started working on. I'll start a new thread on this as his reponses were quite interesting but in particular I noticed that he kept coming back to head down WITHOUT the leg pawing. It's almost as though he uses an "old behaviour" as a comfort zone to retreat to when he's thinking through something new.
Dee, I also find scratches work well and have done quite a bit of clicker+scratch work. This came about last summer as Jak LOVES being scratched in the summer and started getting very bargy in his requests. So I just started asking for a step backwards rather than a barge into me and he ended up progressing to 4-6 steps backwards and then walking nicely forwards again, without barging. This became his cue to ask me for a scratch. I've found that I can also use treats instead of scratches and get the same behaviour - just depends on the occasion as to what he feels like working for. During the winter it tends to be food! But I'm not sure I consider switching from food to scratch as necessarily fading out the treat. I think I know what you mean about a scratch being a secondary reinforcer, but surely depending on the horse and the occasion the horse might consider the scratch to be more valuable than the treat in which case you are doing whatever the opposite of fading out the treat is!? Jak has certainly never needed a food association before enjoying his scratches
Ok it's Emma from CoH here and I have not got a clue if this will work, took me half an hour to find a login name not already in use! Just got a couple of things to say (don't hold your breath, not exciting) you guys have covered everything, but just going to send this to see it if I can work it, before anything else.
What a great thread, it has been so enjoyable to read!
As I said I have some thoughts which may or may not be relevant, but they helped me when I was (and continue to) learn.
VSR's are only a minefield if you get bogged down with the theory. It is important to understand the reason for using them i.e. they are allowing you to train a behaviour to the point you can put it on a cue and faze out the clicker.
As mentioned there are different ways of using a VSR, what is important is finding the way that suits you and your horse. If something you are doing is working, it does not matter if there is another way of doing it, stick with what is working.
If the association of the click (meaning yes) = reward for the horse is trained correctly the horse will start to work for the click. The YES for the horse can be just as rewarding as the reward it's self, hence some of you are finding, your horses responding differently from each other. Don't think you are doing something wrong because other horses are working differently.
There is only so much theory can teach you, the rest is up to natural ability which comes with practice, patience and making mistakes. However your horses will constantly let you know how you are doing. If at all possible try and video your sessions because when you watch them back you will see a lot more than you will while you are doing the work. Watch your horses behaviour and your timing.
Learn to distinguish between their frustration and boredom, if either show up, change what you are doing, remember you are working with their intelligence which is the motor for the behaviour.
Listen to your horse, tune in to them, work on your feel timing and balance.
One of the joys of working with +R is the horse is offering behaviours and we are shaping what is being offered, however this means we are a step behind the horse. Our brain needs to catch up and work with what they are offering even if it was not planned. Look at it like a dance, it works because the two of you are in harmony. Try not to get set in your thinking, go with the flow of the moment. Be open to hear what your horse is telling you and learn from it.
Some horses are bold, some shy, some quick, some slow, we are the same. However as we are the 'teacher' we take the responsibility if what we teach is not working, so we need to look more to our selves. Tune into your instincts.
I know what I am saying is vague, but reading this thread you know the theory so the rest is just to remind you not to forget the pure joy of learning and getting a 'feel' for the behaviours you are working with.
Just to pick upon a couple of facts:
1. The need to reward a horse when you click in the wrong place will depend on where the horse is in its training and your ability to 'read' the situation correctly. If in doubt reward, remembering the horse will repeat what it has been rewarded for, just don't click it again for that behaviour, ignore the frustration and reward something you do want as soon as possible to get back on track.
2.In time with CT you should be able to click without a reward, because the horse is working for the 'yes', but the time it takes to get to this point will depend on your ability to help the horses understand and your ability to use VSR. For instance, I can now long line our pony with clicks only giving him a jackpot at the end. I am not saying go out and do this (it has taken a long while to get good enough with the clicker to use it like this)it is just an example that it can be done.
3. In the beginning the sound of the click will stop a behaviour because as you set up the association you are teaching the horse click means reward. Hence it has to stop what it is doing physically get the treat or rub. However as you move on in your ability with the clicker it is the "click" the horse will be working for, hence you can move on to a VSR. Remember if they have the intelligence to learn the association they are capable of learning VSR,it is how good we are at using it that is important.
Well I have rambled on far too long and intruded on your thread, however it was with the best intentions and if I have thrown up questions I will look back in a couple of days time and do my best to answer them.
after 3 years! I have always been under the impression that the click means a treat of some sort, that is the bargain that you make with your horse. Variable schedules of reinforcement I took to mean that you lengthened the time you clicked so that the horse was offering more of the behaviour that you wanted, i.e. you want to canter all the way round the school instead of a couple of steps, so you prolong the time before you click to keep the canter going. Not all at once but increasing the time gradually. I also always thought that you didn't get onto VSR until the horse really understood the desired behaviour and all that VSR was doing was to gradually wean the horse off the clicker for that behaviour. Therefore you can't use VSR whilst the horse is learning that behaviour as, this is where me Mark Rashid comes in, you are looking for the smallest try and shaping to that behaviour gradually and incrementally but consistently.
Thanks of responding Emma - you're certainly are not rambling!
I think you have managed to sort somthing out in my head. I've been worried because I use treatless clicks with Crunchie as soon as I can. However, I have been doubting the use of treatless clicks and going back to treating after each click. Since doing this I have noticed a drastic change in Cruchie's bhr - biting and being very very pushy for treats (at all times not just in a CT session). I was wondering if it was because of my change of tactics, and I think I may be right- as you say it all depends on your horses personality as the treatless clicks were working fine with Crunch.
It is interesting that you say you can long rein your pony with treatless clicks and only jackpot at the end. This is what I was aiming for with Crunchie. I was using clickless treats but he kept stopping when I clicked. He would start walking again and the durations of the stop seemed to be reducing. But I then doubted my methods and went to treating every click - ummm...think I might go back to the first method in a few weeks time when he has had a break.
It is so easy to doubt yourself, even if something feels right! I'll have to learn to go more with my gut feeling than what is written in a book! I'm sure that this bhr in Crunch is through frustration than anything else- probably brought on by my change in tactics (hopefully I won't case to much of a problem reverting back to my old methods!).
Something that I wondered, it was mentioned on here that treatless clicks are theoretically + punishment (or was it -punishment? or maybe -R...ahhh!)but I am sure this is only in it's mildest form in the context of the clicks. As horses don't actually know the scientific definition of these training methods, would +ve punishment or -R in this context really harm a horse...please don't think I am advocating punishment of -R!!! All I mean is, can science go too far with it's definitions?
Having just read through my posts again I have just realised that I wrote this about Crunchie and treating each click:
'Becasue Crunchie is very bold, if I gave him a treat each time I clicked (forever) I think I would create a very 'in your face' and probably biting horse - I suppose I've managed to get to the point where he listens for the click on the off chance he might get a treat, if he doesn't get a treat he does the bhr again but usually more enthusiastic (sometimes too enthusiastic!).'
Looks like I need to take a little bit of notice of what I say!!!!! I can't beleive that I wrote this and then went and did it....and then got what I warned you all I'd get...
Emma, please ramble more - you have a way with words
Marie, if your horse has learned that the click means "yes" and that has become a reward in itself as Emma said, then a treatless click won't be punishment.
I found this article on VSR in a dog site. Go to www.clickersolutions.com.....click on training articles then scroll down to 'Reinforcement. Punishment @ Extinction'. Go to VSR explained. There is a copyright thingy at the bottom so I didn't send it to the group.
Thanks for your post Emma - if this is what you call intruding, please come and intrude more often! Things are much clearer now, and you have reminded me of a very important principle - I'm allowed to make mistakes I don't have a problem with Mali's mistakes - in fact she doesn't make mistakes, she's just figuring things out. But I sometimes forget that I'm still figuring things out too! Note to self: be a bit nicer to yourself!
Marie, your post really made me laugh. How often do we actually have the answer, or already know the answer, but forget to listen to ourselves? I'm glad I'm not the only one!
Yup, please continue with both!! Thanks, Emma. Great to see you here and thanks for your long post. It really helps get things back in perspective. Also very reassuring as a few days ago I had visions of you reading it whilst sitting with your head in your hands and groaning at us! So it's a relief to know we were on the right lines!
I don't think I even have any questions! Just need to sit quietly and have a think!
Sutton - I think what you are saying is still correct. The click does mean that the treat is coming as if you were to stop treating altogether the click would gradually lose its meaning again. And you do want to get the behaviour established before using VSR but then you use the VSR to improve the behaviour further and you keep on going. And the treatless clicks are the "rewarding the tries" as you shape the behaviour before the final jackpot.
So if you could canter a couple of steps you would stretch it out to 3 steps, then 4, then maybe back to 2 etc and gradually build up. Having treatless clicks there is a way of saying "keep going, treat is on the way", as opposed to no clicks or treats which isn't really giving your horse any information. Eventually when the behaviour has evolved to "canter around the ring" then you would be able to withold the click until after the whole lap.
Hello everyone, my name is Helen, I am a friend of Dees (we did Heathers course together). I hope you don't mind me adding my tuppence worth.
This is the only thread I have had time to read so far, but what a fascinating one it is!! It is so nice to read something where everyone is open. I joined the clickryder dressage group before Christmas and was horrified at how defensive everyone was- I left it within two days! So here I am now. I don't want to tread on any ones toes, I think it is fantastic that this type of discussion is going on, so please please no-one take offense if I say something abit different!
I have to say that on click and treat/ no treat subject I am with Dee and Sam. Everything I learnt in my Psychology degree says that there is no need to click without treating. My interpretation of it is that when you are shaping the behaviour you withhold the click/treat as you gradually mold the behaviour. If you do this gradually enough you will never have a confused animal, because there is always some teeny change that you can click for, even if it means taking a slightly roundabout route to your goal. Once you have reached your goal behaviour, be it hold the foot in the air for 10 secs or back up ten paces or whatever, then you introduce your cue. When you have the behaviour on cue, you phase out the click/treat using your VSR.
I am not saying that any of you are wrong- it is perfectly possible to click without treating without doing any harm (provided you don't do it too often, as obviously too many clicks without treats lead to extinction of classical conditioning), but as a purist I just don't feel it is necessary! Everyone has different ways of working and every horse does too- you should do what works best with you and your animal, as Emma so rightly says, theory is only the beginning, the rest of it is feel and experience!
So there we go, I just think it is great that we are all clicking at all...... just keep spreading the word!
All the best,
Helen
Please do join in, all opinions are very valuable as even if we do not quite agree on something now (and let's face it, these are pretty nit-picky differences of opinion!) it is really useful to have other opinions to draw on if we get stuck at a later date.
Was it you who posted on the clickryder dressage list about bitless bridles? I was also quite shocked at the response you got. My apologies for not commenting at the time but I'm afraid I really don't have time to do more than lurk on the the clickryder lists! Great to have you here instead and feel free to post your bitless bridle message again (afraid I can't remember the details)!
Emma - please could you tell us what made you start doing clickless treats if the majority of people seem to recommend not doing them? Was this trial and error on your part or is it common in other circles?
[Ed - ooops, that should say treatless clicks, not clickless treats!]
This message has been edited by Brocksopp on Feb 5, 2003 3:14 PM
Catherine, to answer your question, I need to go round the houses because I think it is important to explain why I think like I do as opposed to give an answer with no effort on my part of explaining how it is reached..
My interest in equine behaviour in particular began with the questions 'why'? why do horses kick, bite, and rear. What are the differences between the horse that does what we ask and the horse that does not, or raises objection? How do you change a horse’s mind to 'want to' change it's behaviour.
To begin to answer my questions I cleared my mind of what I had been taught by trainers (conventional & 'natural') and started listening to the horses and studying their behaviour. To begin with, I learnt to distinguish between innate and learnt behaviour and then concentrate on aspects that caused differences in behaviour ie. learning speeds, decision making processes and different personalities. During this time, I was coming to understand equine intelligence a subject as a student I had never been encouraged to consider. I was thrilled and excited to watch and learn how horses 'avoided' doing what humans wanted and how they could turn a situation to their advantage. Commonly labelled as 'bad' behaviour, to me it was sheer brilliance and I admire 'bad' or as I call it intelligent behaviour to this day!
From 1995 to 1999 I spent my time observing heard behaviour in America, France and England. During those five years, I was also working with horses from unhandled to highly trained, pet to competitor. Remedial behaviour caused by human error in training and remedial behaviour caused by abuse or neglect. My work and studies also included mules and donkeys.
During these five years I wanted to compare what I was learning from the horses up against the science of behaviour and psychology. To do this I also studied the work of trainers of other species namely marine and bird because I wanted to understand in greater depth working with animal intelligence. This led me to study applied behavioural science which led to Operant Conditioning and in particular Clicker Training.
These are my conclusions, 'today' (as I am the first to admit I am a constant work in progress, the day I stop learning is the day I no longer belong working with horses).
Understanding, working with and studying the science of behaviour has allowed me to know 'how' and 'why' based on academic research and conclusion. It has given me a platform from which to base the facts of behaviour and psychology. However, the majority of theory has been based on research and experiments done under controlled circumstances and or environments. This is needed to get accurate measurements of behaviour. However, in any measurement of behaviour there are always variables, the unaccounted for, or unwanted behaviours that don't fit. For instance in a group of 10 pigeons 9 will hit a leaver to get grain, 1 would rather preen its feathers. In measuring behaviour or compartmentalising psychology, you need the majority to prove theory. It is essential knowledge and work but it is not the B and end all of working with behaviour and this is why in the real world of working with horses we cannot rely on theory alone because it does not (in general) account for the variables.
This is why there is more than one way of working with VSR. The facts of science are just that, facts. They are a fantastic platform for us to base our understanding, but as trainers working outside of laboratories or controlled environments we have to be able to accommodate what is happening on a daily basis, incorporating all the variables. This is why I personally do not believe any trainer should ever say, one particular way is the best, or only way because it will not work for everyone or every horse, let alone the unique combination of the two.
I am of the same opinion in all areas take psychologists for example. Some practice under the banner of Freud and some Jung, surly by practising or understanding both you help more people because you have more guidance to offer depending on the need of the patient.
So in 1999 Ben and I set up Company of Horses designed to bridge the gaps between the academics of behaviour and the work we do at home with our horses. The practical application of the science of behaviour, with the practical ability to work with the variables that do not fit the mould or follow the rules. However, where we and any trainer will always fall short is we cannot teach innate ability. That comes from within every individual and is a gift. It is that gift that will be the difference between a great trainer and a great book!
So Catherine, in an exceptionaly long winded reply, I have worked with so many horses that throw the 'rule' book out of the window that I have had to learn to work based on an individuals learning and motivations and yes this has involved a lot of 'trail & error'learning.
The pony I mentioned Started off with the association click means treat, then I was going to a text book VSR, very slow, very steady, with the plan to introduce a cue and eventualy remove click and treat. However Harry had other idears! he is basicaly unhandled and we started with targets, I palnned he would touch it and slowly would train him to pick it up. I only clicked & rewarded a couple of times, for touching it, when he picked it up an gave it to me. I put the whole thing on a cue and we missed out VSR all togeather. I wanted to teach him to pick up his feet. I did each one, click and reward and in the next session, I picked up one, clicked, and before I could reward he had picked up the next, same for the other two and he got a jackpot at the end. Again no VSR. There is not doubt he wants the carrot, but he will offer behaviours because he 'get's it' the click for him is such a big yes that he keeps going.
But this is Harry and the speed he works at is daunting. I had to learn to go at his pace, because there was no way I was going to try and slow him down to accomadate me.
I have a mare here who needs to go slowly with a VSR and another who is an expert in training me and will slip us into a pattern with a VSR to her constant advantage if I am not on the ball.
So clicks and reward and clicks and no reward will depend totally on the individual you are working with, so will the speed, duration of lessons and your shaping plan. We do not all learn the same way, neither do our fantstic horses. It makes our job harder because we cannot just pigeon hole everything (thank God) and look what they teach us ))
I truly hope there are lots of people working with different variations of VSR, otherwise how can we accomadate the variables?
I have been trying to get back onto the forum since I posted the last message but for whatever reasons I kept getting a bad gateway. I had composed a reply in my head which I have since forgotten but it was something to the effect of don't get hung up on theory just do what you do because there are a million routes of getting where you want to go, some of them go at different paces but they all take you there! Emma has said all this a million times better than I ever could so I won't bother trying! All I will add is that the difference in having innate ability as a trainer and just having book learning is that the former will always be able to chose the shortest route for that particular animal. Think i will always be working towards that!
And yes, it was my post about the bitless bridle: I will put it on here for you all to read!
Take care,
Helenx
Thankyou all for your kind words and support,I may be a 'trainer'but I have not lost touch with my first passion which is loving horses and at the end of the day I am just an owner with the same concerns and questions as everyone else, I just talk a lot!!
I am very aware that my contributions detracted from Sams question and following conversation. Please don't stop. As long as the motivation is grounded I am a true believer that there is no right or wrong only learning. Who knows, as long as we all keep talking and asking questions we may just find new ways, undiscovered ways of understanding and working with horses. Who knows maybe someone from this forum will re write the 'rule book'
Interested in reading all the above. Because I drive my horses - when using VSR I tend to add my voice as the marker (you're are going in the right direction cue) instead of just the click. This is because when driving the voice is one of the most important aids. What is so refreshing about this thread is that nothing is set in stone and we are learning all the time from each other (and its making me think!!)
..Emma, it's certainly not because of your contributions that I've stopped adding to this thread. I really need to get on and do it now. So far I've been trying to get my head round VSR theory without putting enough of it into practice.
I moved to a new yard a couple of days ago and so the most I'll be doing over the next couple of weeks is really basic stuff that we can already do. But in a few weeks' time when we're ready to get back to the VSR I'm sure I'll have a load more questions/comments. And as the new yard is mud-free and has a schooling ring we should have more potential behaviours to choose from...!
Current Topic - Variable Schedules of Reinforcement
