How to deal with condor symlink error in new Rocks

For errors like the following:

/usr/lib/ruby/1.8/fileutils.rb:338:in `symlink': File exists - /afs/cs.wisc.edu/p/multifacet/users/wang/rocks_base/new_rocks/gem5/build/LIBRUBY_MOESI_CMP_directory/libm5_fast.so or /afs/cs.wisc.edu/p/multifacet/users/wang/rocks_base/new_rocks/gem5/build/LIBRUBY_MOESI_CMP_directory/ruby.so (Errno::EEXIST)
        from /usr/lib/ruby/1.8/fileutils.rb:338:in `ln_s'
        from /usr/lib/ruby/1.8/fileutils.rb:1395:in `fu_each_src_dest0'
        from /usr/lib/ruby/1.8/fileutils.rb:336:in `ln_s'
        from /usr/lib/ruby/1.8/fileutils.rb:359:in `ln_sf'
        from /afs/cs.wisc.edu/p/multifacet/users/wang/rocks_base/new_rocks/rocks:145
/usr/bin/ruby: symbol lookup error: /afs/cs.wisc.edu/p/multifacet/users/wang/rocks_base/new_rocks/bochs/bx-ruby-fast/bochs.so: undefined symbol: _Z15libruby_destroyv

I need to add two lines in the condor.rb file:

              softlink_cmd = "ln -s #{root}/../gem5/build/LIBRUBY_MESI_CMP_directory/libm5_fast.so #{workload_output}/ruby.so"
              rm_softlink = "rm -f #{workload_output}/ruby.so"
              puts rm_softlink
              puts softlink_cmd
              system(rm_softlink)
              system(softlink_cmd)

How does the NEW version of Rocks config works?

now, everything is moved into the $NEW_ROCKS/config/ directory
rocks_cfgs/HomoCMPTwoLevelInorder.rb calls ruby_user_cfgs/TwoLevel_SplitL1UnifiedL2.rb

These script files are also duplicated in the old config directory. Need to be aware which files are modified.

gdb script

to execute a gdb script:
gdb -x script_file
This will allow gdb to execute the specified script_file. More flexible than the .gdbinit option.

In the script, functions can be defined:

def test_case1
b main
end

test_case1

Some notes on Rocks and how Ruby cycle is bumped

end_cycle()  in bx_ruby_main.cc gives up the control from the current cpu and then the system calls choose_next_thread and setup the stack for the next cpu to run.

choose_next_thread in bx_ruby_main.cc try to find a cpu that is not stalled and has some work to do.

cur_ruby_cycle in bx_ruby_main.cc counts the number of ruby cycles executed.

calling end_cycle until the current_ruby_cycle is the same as the some future time, this allows a particular operation to take 10 cycles. Shown in the following:
    int new_time = cur_ruby_cycle + 10;
    while(new_time != cur_ruby_cycle) {
      BX_INFO(("waiting the current ruby cycle %ld", cur_ruby_cycle));
      end_cycle();
    }

How does the ruby_hitcallback work in Rocks?

ruby_hitcall is the primary method that Ruby communicates with the Bochs system. When Bochs issues a memory request to Ruby, it set up the ruby_hitcallback through the libruby_get_port in $ROCKS/bochs/cpu/init.cc
/**

*  Retrieve a handle to a RubyPort object, identified by name in the
*  configuration.  You also pass in the callback function you want
*  this port to use when a request completes.  Only one handle to a
*  port is allowed at a time.
*/

RubyPortHandle libruby_get_port(const char* name, void (*hit_callback)(int64_t access_id));
This RubyPortHandle is part of the CPU object in bochs. It is defined in $ROCKS/bochs/cpu/cpu.h:793.
When libruby_issue_request is called, the Ruby_PortHandle is passed alone with the RubyRequest Struct. 
The Ruby_PortHandle is defined in
gem5/src/mem/ruby/libruby.hh:8
typedef void* RubyPortHandle;

This pointer is set up in the configuration time through the rocks script. The RubySystem is defined in the $ROCKS/gem5/src/mem/ruby/System.hh and it is just a big container that has everything.

Ruby_PortHandle is casted into RubyPort in libruby_issue_request at libruby.cc.

RubyPort class is in system/RubyPort.hh and other classes inherit from this class. Important ones are Sequencer class and DMASequencer class.

And in RubyPort class, one could make a request. This request can be then a memory request.

Who calls this Ruby_hitcallback? Ruby or Bochs? What is Bochs role? What is Ruby's Role?

In essence, the bochs register a call back for Ruby to call when ruby is done. And Ruby side port, such as sequencer will call the callback function when it is done. The point of the callback is stored in Sequencer, which is child class of RubyPort. 

This is a short description that I wrote on the wiki:
ruby_hitcall, defined in $ROCKS/bx_ruby_interface/bx_ruby_memory.cc is the primary method that Ruby communicates with the Bochs system. When Bochs initiates at the beginning of everything, Bochs sets up ports in the M5 fashion to communicate with Ruby. These ports will allow Bochs to issues memory request to Ruby. When the port was set up through the libruby_get_port in $ROCKS/bochs/cpu/init.cc, a function pointer is passed to the RubyPort object, stored as m_hit_callback. The definition of the RubyPort object can be found at $ROCKS/gem5/src/mem/ruby/system/RubyPort.hh/cc. The primary object in Ruby that receives the memory requests are the Sequencer object and the DMASequencer object. They are both defined in the $ROCKS/gem5/src/mem/ruby/system and both inherits from the RubyPort class. In the member function Sequencer::hitCallback, the function pointer is used by the Sequencer to call Bochs. This function pointer in Sequencer will call ruby_hitcallback.The implementation for DMASequencer is in a similar fashion.

How to get callgrind to work in older Rocks

in rocks the ruby script that fires the simulation
search for valgrind and at the end of the file,

elsif @@valgrind # uses fast bochs
        system("valgrind --leak-check=yes "+@@root+"/bochs/"+BochsConfig.binName +
               " "+BochsConfig.getCommandLineOptions)
This will have the valgrind work with leak-check

but if we change the leak-chcek with --tool=callgrind, it should work with callgrind.

To see the call graph, use
kcachegrind callgrind.out.14912

kcachegrind is already installed in the csl machines.

AFS condor setup

Chagnes for LibRuby_clear_stats

diff -r 971902a8740e src/mem/ruby/system/System.cc
--- a/src/mem/ruby/system/System.cc Tue Oct 20 15:29:02 2009 -0500
+++ b/src/mem/ruby/system/System.cc Mon Mar 01 20:28:19 2010 -0600
@@ -345,7 +345,7 @@
}
}

-void RubySystem::clearStats() const
+void RubySystem::clearStats()
{
m_profiler_ptr->clearStats();
m_network_ptr->clearStats();
diff -r 971902a8740e src/mem/ruby/system/System.hh
--- a/src/mem/ruby/system/System.hh Tue Oct 20 15:29:02 2009 -0500
+++ b/src/mem/ruby/system/System.hh Mon Mar 01 20:28:19 2010 -0600
@@ -129,7 +129,7 @@
void recordCacheContents(CacheRecorder& tr) const;
static void printConfig(ostream& out);
static void printStats(ostream& out);
- void clearStats() const;
+ static void clearStats() ;

uint64 getInstructionCount(int thread) { return 1; }
static uint64 getCycleCount(int thread) { return g_eventQueue_ptr->getTime(); }

Set Fully Associtive Cache with LRU

I need to change CacheMemory.hh to enable fully associtive cache with LRU in Rocks.

ale-01% hg diff CacheMemory.hh


diff -r 971902a8740e src/mem/ruby/system/CacheMemory.hh
--- a/src/mem/ruby/system/CacheMemory.hh Tue Oct 20 15:29:02 2009 -0500
+++ b/src/mem/ruby/system/CacheMemory.hh Mon Feb 22 20:42:01 2010 -0600
@@ -218,8 +218,19 @@
}
}

+
+
m_cache_num_sets = cache_size / m_cache_assoc;
- m_cache_num_set_bits = log_int(m_cache_num_sets);
+ if(m_cache_assoc > cache_size) {
+ // this is for full assoc
+ m_cache_num_set_bits = 0;
+ m_cache_num_sets = 1;
+ } else {
+ m_cache_num_set_bits = log_int(m_cache_num_sets);
+ }
+

if(policy == "PSEUDO_LRU")
m_replacementPolicy_ptr = new PseudoLRUPolicy(m_cache_num_sets, m_cache_assoc);
@@ -277,7 +288,18 @@
Index CacheMemory::addressToCacheSet(const Address& address) const
{
assert(address == line_address(address));
- return address.bitSelect(RubySystem::getBlockSizeBits(), RubySystem::getBlockSizeBits() + m_cache_num_set_bits-1);
+ if(m_cache_num_set_bits > 0 ) {
+ return address.bitSelect(RubySystem::getBlockSizeBits(), RubySystem::getBlockSizeBits() + m_cache_num_set_bits-1);
+ } else if(m_cache_num_set_bits == 0 ) {
+ // This is the case that not cache set bit is used
+ // Because this is the fully assoicitive cache
+ //return address.bitSelect(RubySystem::getBlockSizeBits(), RubySystem::getBlockSizeBits() + m_cache_num_set_bits);
+ return 0;
+ } else {
+ // there is something wrong here that m_cache_num_set_bits is less than 0 ?!
+ assert(1!=0);
+ return -1;
+ }
}